Latency budgets and critical-path analysis

Orientation

Definition

A latency budget divides a user-facing latency target among the steps that make up a request. Critical-path analysis finds the sequence of steps that actually determines total time: the longest path through the request's call graph. Only time saved on that path shortens the request.

Why it matters

Teams spend much of their performance effort on the wrong things. In a parallel stage, speeding up anything but the slowest branch saves nothing at all. Without budgets, every team adds "just 10 ms", and a product slowly misses its target with no one responsible. A budget makes latency a shared, allocated resource, gives each team a number it can own, and provides the basis for sensible timeouts. A critical-path view also tells you what's worth funding: it shows a precise saving before anyone has written any code.

Prerequisites

Latency, P99 latency

Study time

About 1.5 hours, including the terminal lab and challenges. This is an estimate.

Related topics

Timeouts (not yet written), Deadlines and cancellation propagation (not yet written), Tail latency, Latency

Learning objectives

• Build a call graph, then find its critical path and each step's slack.
• Predict the saving from speeding up a step, including when it is zero.
• Allocate a budget across steps, with reserves for retries and growth.
• Explain why percentile budgets don't simply add, and when correlation breaks your assumptions.
• Derive timeouts from a budget, and propagate deadlines.

Production scenario

A checkout must return in 300 ms at p99, and it currently takes about 265 ms in the median trace with occasional misses. Three teams volunteer optimizations. The pricing team can cut pricing from 40 ms to 10 ms with a new cache, at a cost of two engineer-months. Critical-path analysis shows that pricing runs in parallel with fraud and inventory, and fraud takes 90 ms. Pricing has 50 ms of slack, so the project would save exactly 0 ms.

The fraud team offers to go from 90 ms to 40 ms. That's on the critical path, but inventory at 60 ms becomes the new longest branch once fraud drops below it, so the saving is 30 ms, not 50 ms. The largest step is payment at 120 ms, a third-party call which can't be sped up, though it might be moved after the response using an asynchronous confirmation. The team's plan changes completely, using only a call graph and some arithmetic. This lesson's TypeScript example computes these figures.

Mental model

Analogy

Think of preparing a dinner. The roast takes 90 minutes, the potatoes take 40, and the salad takes 10. All must be ready at once, so dinner takes 90 minutes. Buying a faster salad spinner doesn't serve dinner any sooner. That's the critical path: the roast. Once the roast is faster than the potatoes, they become the new thing you're waiting for. Budgeting says how long each dish may take if dinner has to be on the table by 7.

Where the analogy breaks

Cooking times are predictable, while service latencies are distributions: the path that's critical changes from request to request, so one request's critical path may not match another's. Cooks can also watch a clock and adapt, while a service needs a deadline passed to it explicitly. A pot also doesn't get slower because others are cooking, whereas services often slow each other through shared resources.

Entities, state, flow, and invariants

• Entities: steps, dependencies, budgets, deadlines, and traces.
• State: remaining budget as a request moves through the system.
• Flow: user target, allocation by step, per-call timeout, runtime deadline propagation, and measurement against budget.
• Invariant 1: request time equals the longest path through the graph.
• Invariant 2: a step with slack can slow down by that much with no effect on the request.
• Invariant 3: no call waits longer than the remaining budget.

Foundations

Concepts and terminology

First-principles explanation

Sequential steps add and parallel steps take their maximum, so request duration is the longest path. Compute it with a forward pass (each step's earliest finish equals its duration plus the latest dependency's finish) and a backward pass (the latest finish that doesn't delay the total). Slack is their difference. Steps with zero slack form the critical path. This is the classic Critical Path Method, developed for project scheduling in the 1950s, and it applies directly to requests.

There's a catch that matters a great deal. Reducing a critical step only saves time until another path becomes longest. So savings are capped by the gap to the next-longest path. In the example, fraud leads inventory by 30 ms, so that's the most a fraud optimization alone can save.

Budgets face a second catch: percentiles don't add. If delays are independent, it's unlikely that several steps are slow together, so the true p99 of a sum is less than the sum of p99s. In the Python example, the p99s total 451.9 ms, while the sequential p99 is only 311.2 ms. But if steps slow down together, due to a shared host or network, the p99 reaches 559.3 ms, above the naive sum. So a sum of p99s is neither a safe upper bound nor a good estimate. Measure the end-to-end percentile using traces, and treat per-step budgets as allocations for accountability.

Mathematics

Forward pass: EF(s) = duration(s) + max(EF(d)) over all dependencies d.

Backward pass: LF(s) = min(LF(t) − duration(t)) over all successors t, with LF = total for steps that have no successors.

Slack: LF(s) − EF(s).

Checkout worked example: cart finishes at 5 + 10 + 25 = 40 ms. Then inventory finishes at 100, pricing at 80, and fraud at 130. Payment starts at 130 and finishes at 250, and writing the order finishes at 265. Pricing slack: 130 − 80 = 50 ms. Inventory slack: 30 ms.

Measured percentile combination, from a 100,000-trial simulation:

Timeout from budget: remaining = deadline − now − reserve for the rest of the path.

Complexity

The forward and backward passes visit every step and edge, so an efficient version is O(V + E). The example scans successors linearly for clarity, giving O(V × (V + E)) in the worst case, which is fine for a few dozen steps. Monte Carlo costs O(n × V) for n trials. Across real traces, critical paths must be computed per trace, then aggregated.

Mechanism

Normal path

1. Choose a journey and target: POST /checkout, p99 300 ms.
2. Build the call graph from traces, not from memory or diagrams.
3. Compute critical paths across many sampled traces, since the critical path varies.
4. Set aside a reserve, commonly 10% to 20%, for network hops and growth.
5. Allocate the rest to steps, based on what each can achieve.
6. Derive timeouts from allocations, and propagate a deadline along the path.
7. Report each team's use of its budget, then review changes to the budget.

State transitions

stateDiagram-v2
    [*] --> BudgetFull: request received
    BudgetFull --> Spending: step begins
    Spending --> BudgetRemains: step finished, time left
    BudgetRemains --> Spending: next step begins
    BudgetRemains --> Degraded: too little left for optional step
    Degraded --> Spending: required step continues
    Spending --> FailedFast: deadline reached
    BudgetRemains --> Done: path complete
    Done --> [*]
    FailedFast --> [*]

The Degraded state is what makes budgets useful at runtime. With only 20 ms left, a service can skip recommendations and still complete the essential work. Without a deadline in hand, it can't know to do that. See Deadlines and cancellation propagation (not yet written).

Concurrency behavior

Parallelizing a sequential chain shortens the path, but only if the steps are truly independent. It also raises concurrent load, connection use, and the odds of meeting a slow outlier. When parallel branches share a resource, such as one database, they queue, and the apparent parallelism is partly fiction. Traces reveal this as spans that overlap on paper while their durations grow. See Tail latency.

Ordering and consistency

Some steps have to be ordered: inventory reservation must precede payment capture to avoid charging for items you can't ship. That's a correctness constraint, so it limits how much parallelization is possible. When reviewing a graph, ask of each dependency edge if it reflects a data dependency, a correctness rule, or just the order someone wrote the code in. The last kind is often free to remove.

Ownership of data and control

The journey owner owns the overall target and allocation. Each service team owns its budget and may trade it through negotiation. The platform team owns the deadline propagation mechanism. Budget owners should be named, or budgets decay. Third-party calls need special treatment, since you own the timeout but not the latency.

Architecture

flowchart LR
    C["Client"] -->|"HTTPS, target p99 300 ms"| G["Gateway: 5 ms"]
    G --> A["Auth: 10 ms"]
    A --> CA["Cart: 25 ms, owns carts"]
    CA -->|"gRPC, parallel"| I["Inventory: 60 ms, owns stock"]
    CA -->|"gRPC, parallel"| P["Pricing: 40 ms"]
    CA -->|"gRPC, parallel"| F["Fraud: 90 ms, on critical path"]
    I --> PAY
    P --> PAY
    F --> PAY
    subgraph ext["External provider: outside our control"]
      PAY["Payment: 120 ms"]
    end
    PAY --> W["Order write: 15 ms, owns orders"]

Fraud determines when payment can begin, so it, not inventory or pricing, lies on the critical path. The external boundary around payment means the 40% of the budget that it takes isn't in your control, which suggests a different design question: must the customer wait for payment capture at all?

flowchart LR
    subgraph ts["TypeScript: fixed durations"]
      S["Step list"] --> TO["topoSort: rejects cycles"]
      TO --> FB["Forward and backward passes"]
      FB --> CP["Critical path and slack"]
      FB --> WI["whatIf: saving from a change"]
    end
    subgraph py["Python: distributions"]
      D["Dependency distributions"] --> MC["Monte Carlo: 100,000 trials"]
      MC --> R["Sum vs independent vs correlated vs parallel"]
      MC --> DL["Deadline: success rate and time spent"]
    end

The two examples address different questions. Fixed durations are clear for reasoning about structure, while distributions show what really happens to percentiles. Use the first to find what to work on, and the second to avoid over-trusting budget arithmetic.

sequenceDiagram
    participant G as Gateway
    participant C as Cart
    participant F as Fraud
    participant P as Payment provider
    G->>C: Deadline: 295 ms remaining
    C->>F: Deadline: 255 ms remaining
    Note over F: Enough time for the full model
    F-->>C: Score after 90 ms
    C->>P: Remaining 165 ms, reserve 15 ms for write
    P-->>C: Authorized after 120 ms
    C-->>G: Complete with 30 ms to spare

The budget is passed as a value, so each hop can make local decisions. Timeouts for the final dependency are derived from what's actually left, after reserving time for later steps, and not from a static configuration value that's unaware of what has already happened.

flowchart TD
    A["Static 1 s timeout at every hop"] --> B["Payment stalls"]
    B --> C["Cart waits 1 s, then retries twice"]
    C --> D["3 s of work for a 300 ms target"]
    D --> E["Client already gave up"]
    E --> F["Connection pool fills with dead work"]
    F --> G["Unrelated requests fail"]
    B -.->|"with propagated deadline"| H["Stops at remaining budget"]
    H -.-> I["Fails fast and frees the connection"]

Static timeouts are chosen without knowing the journey, so they frequently add up to several times the user's patience. The dashed route shows the fix. Note that 1,000 ms timeouts at a 2,000 requests-per-second rate means up to 2,000 calls in flight, compared with 240 in normal conditions (2,000 × 0.12 s). That's why pools fill so quickly. See Timeouts (not yet written).

Trade-offs

Advantages

• Directs effort to changes that help, and prevents those that can't.
• Turns a shared target into something each team can own.
• Provides a principled basis for timeouts.
• Prevents slow decay from many small additions.

Costs

• Requires tracing, and upkeep as graphs change.
• Budgets can cause arguments between teams.
• Allocations are an accounting tool, as percentiles don't add up exactly.
• A strict budget might discourage useful features if it's not negotiable.

Alternatives

• End-to-end SLO only: simple, and leaves no one accountable when it degrades.
• Performance regression tests: catch slow growth, and complement budgets.
• Asynchronous design: remove a step from the critical path altogether. See Synchronous versus asynchronous composition (not yet written).
• Sampling profiler: better within a process, while critical paths work across services.

When to use it

For interactive journeys with several services, for strict SLAs, and whenever a new dependency is proposed for a critical journey.

When not to use it

Don't budget a simple service with one dependency, or batch jobs where total throughput is what matters. Avoid budgeting for exploratory features, where the user value isn't yet proven.

Reconsider when

• The critical path shifts between steps in traces.
• A journey's fan-out or dependencies change.
• A team has been consuming over its budget for two windows.
• The external provider's latency changes.

Capacity planning

Inputs and assumptions

2,000 checkouts per second at peak, a 300 ms p99 target, and the budget in the scenario. Assume a 10 ms cross-zone penalty during zone failure and a 200-connection limit per provider client pool.

Formula and units

• Concurrent calls = rate (requests per second) × duration (seconds), from Little's Law.
• Reserve (ms) = target − sum of the critical path.
• Worst in-flight = rate × timeout.

Intermediate calculations

• Critical path: 5 + 10 + 25 + 90 + 120 + 15 = 265 ms.
• Reserve: 300 − 265 = 35 ms, or 11.7%.
• Payment concurrency: 2,000 × 0.120 s = 240 calls in flight.
• At a 1 s timeout during a stall: 2,000 × 1 s = 2,000 calls in flight.
• At a budget-derived 165 ms timeout: 2,000 × 0.165 s = 330 calls in flight.

Headroom

The 200-connection pool is already short of the normal 240. At 30% headroom: 240 ÷ 0.7 = 343 connections, which also just covers the 330 worst case when using deadlines. A static 1 s timeout would need about 2,000 connections, making the deadline the cheaper and safer choice.

Failure capacity

If a zone fails and calls become cross-zone at 2 ms per hop, then 6 hops add 12 ms, which is under the 35 ms reserve. One retry of a 90 ms fraud call would not fit in it, so retries on the critical path need another budget source, such as dropping an optional step. Payment latency doubling to 240 ms breaks the target by itself, so you need a plan for this: fail fast, or complete asynchronously.

Failure modes

Security and privacy

Threats

Budgets can pressure teams into removing security steps, which is a risk to watch for directly. A client-supplied deadline can also be abused: a very long deadline holds resources, and a very short one wastes work. An attacker who controls a slow upstream, such as a webhook target, can use up your budget. See Server-side request forgery (not yet written).

Trust boundaries

Never trust a client's deadline as is. Cap it at the edge, then propagate your own. Treat partner calls as outside your control and give them strict timeouts.

Authentication and authorization

Auth steps should have a budget rather than be skipped. Decide explicitly whether a timeout in authorization fails open or closed: for most actions, it should fail closed. Cache results within their revocation requirements to reduce critical-path time safely.

Tenant isolation

A heavy tenant can make each step slower for everyone. Track budget use per tenant class, and isolate noisy traffic so that one tenant's slow queries don't consume another tenant's budget. See Tenant isolation and noisy-neighbor controls (not yet written).

Sensitive data

Traces used for critical-path analysis can include identifiers, query text, and payment details, so scrub attributes and restrict access. Don't put card numbers or tokens in span attributes.

Abuse resistance

Skipping optional steps under time pressure must not skip checks: fraud checks in the critical path could be bypassed if an attacker can force a low budget. Mark such steps as mandatory, so that a request fails rather than proceeding without them.

Observability

Metrics

Use a per-step latency histogram labeled by journey, end-to-end journey latency, deadline-exceeded counts by hop, degraded-response counts, and budget consumption, which is the step's p99 divided by its allocation.

Logs

When a deadline is exceeded, log the remaining budget on entry at each hop and the step that used it up. Log what was skipped in degraded responses.

Traces

Traces are the source of truth for this topic. Compute the critical path per trace and aggregate: "fraud was on the critical path in 72% of traces, and contributed an average of 88 ms." See Distributed tracing (not yet written).

Dashboards

Show a stacked view of budget per step against the target, a trend line per step, and how often each step is on the critical path. This shows drift before it breaches the SLO.

Alerts

Page on the journey SLO. Send tickets to the owning team when a step exceeds its budget for several days. Avoid paging on per-step budget, as budgets are accounting and don't directly measure user impact.

SLO examples

• 99% of POST /checkout requests finish within 300 ms over 28 days.
• The fraud step's p99 stays within 90 ms, reviewed weekly.
• Under 0.1% of checkouts end with a deadline-exceeded response.

Runbook actions

When the journey misses its target: open slow traces, find the step that most often sits on the critical path, compare it with its budget, and check for recent changes in that step. If it's a third party, follow the degradation plan.

Implementation

Architecture of the examples

TypeScript computes critical paths and what-if savings for a fixed graph. Python simulates distributions to show how percentiles combine and how deadlines bound work. Neither has an HTTP layer since both are analysis tools. In production, their input would come from your tracing system.

Data structures

Step holds an ID, a duration, and dependencies. analyze uses depth-first topological sorting with visiting and done states, which also catches cycles. Maps keep earliest and latest finish per step. Python uses Pydantic for Dependency and a small frozen dataclass holding the random generator.

Interfaces

analyze(steps) returns { totalMs, criticalPath, steps }, and whatIf(steps, id, newDurationMs) returns before, after, and saved. Python's sequential, parallel, correlated_sequential, and with_deadline each return summary data.

Error handling

GraphError covers duplicate IDs, unknown dependencies, cycles, and an empty graph. A cycle error names the path. Negative durations are rejected, including NaN, through the check !(duration >= 0), which a plain < 0 test would miss.

Concurrency considerations

The analyzer is a pure function, so it needs no synchronization. The Python simulation models parallel execution by taking a maximum, and doesn't actually run anything concurrently. A real implementation would have to use cancellation to stop remaining branches once the deadline passes.

Educational limitations

Fixed durations hide variation. The graph has one shape, whereas real requests branch on cache hits and conditions, so one trace's critical path can differ from another's. The correlated model uses a single shared slowdown.

Production hardening

Compute critical paths from trace data per route, aggregate over many traces, include network gaps as steps, propagate deadlines with gRPC or a header, and publish budget dashboards.

Example source

// Critical-path analysis over a call graph. A request is a directed acyclic graph of steps with
// durations. The critical path is the longest route through it, and slack is how much a step can
// slow down before it delays the request. Only steps with zero slack are worth speeding up first.
import { fileURLToPath } from 'node:url';

export interface Step { id: string; durationMs: number; dependsOn: string[] }
export interface StepTiming { id: string; earliestStart: number; earliestFinish: number; slack: number; critical: boolean }
export interface Analysis { totalMs: number; criticalPath: string[]; steps: StepTiming[] }

export class GraphError extends Error {}

function topoSort(steps: Step[]): Step[] {
  const byId = new Map(steps.map((s) => [s.id, s]));
  if (byId.size !== steps.length) throw new GraphError('Step IDs must be unique.');
  const state = new Map<string, 'visiting' | 'done'>();
  const order: Step[] = [];
  const visit = (s: Step, trail: string[]): void => {
    const seen = state.get(s.id);
    if (seen === 'done') return;
    if (seen === 'visiting') throw new GraphError(`Cycle: ${[...trail, s.id].join(' -> ')}`);
    state.set(s.id, 'visiting');
    for (const d of s.dependsOn) {
      const dep = byId.get(d);
      if (!dep) throw new GraphError(`${s.id} depends on unknown step ${d}.`);
      visit(dep, [...trail, s.id]);
    }
    state.set(s.id, 'done');
    order.push(s);
  };
  for (const s of steps) visit(s, []);
  return order;
}

export function analyze(steps: Step[]): Analysis {
  if (steps.length === 0) throw new GraphError('At least one step is required.');
  for (const s of steps) if (!(s.durationMs >= 0)) throw new GraphError(`${s.id} has an invalid duration.`);
  const order = topoSort(steps);
  const finish = new Map<string, number>();
  const start = new Map<string, number>();
  for (const s of order) {
    const es = Math.max(0, ...s.dependsOn.map((d) => finish.get(d) ?? 0));
    start.set(s.id, es);
    finish.set(s.id, es + s.durationMs);
  }
  const total = Math.max(...finish.values());
  const latest = new Map<string, number>(); // latest allowed finish
  for (const s of [...order].reverse()) {
    const successors = steps.filter((o) => o.dependsOn.includes(s.id));
    const lf = successors.length === 0 ? total : Math.min(...successors.map((o) => (latest.get(o.id) ?? total) - o.durationMs));
    latest.set(s.id, lf);
  }
  const timings: StepTiming[] = order.map((s) => {
    const ef = finish.get(s.id) ?? 0;
    const slack = (latest.get(s.id) ?? total) - ef;
    return { id: s.id, earliestStart: start.get(s.id) ?? 0, earliestFinish: ef, slack, critical: slack === 0 };
  });
  return { totalMs: total, criticalPath: timings.filter((t) => t.critical).map((t) => t.id), steps: timings };
}

// Shows what speeding up one step achieves. The saving is capped by the next-longest path.
export function whatIf(steps: Step[], id: string, newDurationMs: number): { before: number; after: number; savedMs: number } {
  const before = analyze(steps).totalMs;
  const after = analyze(steps.map((s) => (s.id === id ? { ...s, durationMs: newDurationMs } : s))).totalMs;
  return { before, after, savedMs: before - after };
}

export const CHECKOUT: Step[] = [
  { id: 'gateway', durationMs: 5, dependsOn: [] },
  { id: 'auth', durationMs: 10, dependsOn: ['gateway'] },
  { id: 'cart', durationMs: 25, dependsOn: ['auth'] },
  { id: 'inventory', durationMs: 60, dependsOn: ['cart'] },
  { id: 'pricing', durationMs: 40, dependsOn: ['cart'] },
  { id: 'fraud', durationMs: 90, dependsOn: ['cart'] },
  { id: 'payment', durationMs: 120, dependsOn: ['inventory', 'pricing', 'fraud'] },
  { id: 'write-order', durationMs: 15, dependsOn: ['payment'] },
];

if (process.argv[1] === fileURLToPath(import.meta.url)) {
  try {
    const a = analyze(CHECKOUT);
    console.log(JSON.stringify({ totalMs: a.totalMs, criticalPath: a.criticalPath }));
    for (const t of a.steps) console.log(JSON.stringify({ id: t.id, finish: t.earliestFinish, slack: t.slack }));
    console.log(JSON.stringify({ optimizePricingTo10: whatIf(CHECKOUT, 'pricing', 10) }));
    console.log(JSON.stringify({ optimizeFraudTo40: whatIf(CHECKOUT, 'fraud', 40) }));
  } catch (err) {
    console.error(JSON.stringify({ level: 'error', msg: 'analysis.failed', error: String(err) }));
    process.exitCode = 1;
  }
}

"""Why percentile budgets do not simply add up, shown by seeded Monte Carlo simulation.

Each dependency has its own latency distribution. Adding their p99s overstates sequential total p99
when the delays are independent, and understates the effect of parallel branches. Timeouts should be
set from the remaining budget, so this also shows how a deadline protects the total.
"""

from __future__ import annotations

import json
import math
import random
from dataclasses import dataclass

from pydantic import BaseModel, Field


class Dependency(BaseModel):
    name: str
    median_ms: float = Field(gt=0)
    sigma: float = Field(gt=0)


class Summary(BaseModel):
    p50_ms: float
    p99_ms: float


@dataclass(frozen=True)
class Sampler:
    rng: random.Random

    def draw(self, d: Dependency) -> float:
        return self.rng.lognormvariate(math.log(d.median_ms), d.sigma)


def percentile(values: list[float], q: float) -> float:
    ordered = sorted(values)
    return ordered[math.ceil(q * len(ordered)) - 1]


def summarize(values: list[float]) -> Summary:
    return Summary(p50_ms=round(percentile(values, 0.5), 1), p99_ms=round(percentile(values, 0.99), 1))


def own_p99(d: Dependency, n: int, seed: int) -> float:
    s = Sampler(random.Random(seed))
    return percentile([s.draw(d) for _ in range(n)], 0.99)


def sequential(deps: list[Dependency], n: int, seed: int = 1) -> Summary:
    s = Sampler(random.Random(seed))
    return summarize([sum(s.draw(d) for d in deps) for _ in range(n)])


def parallel(deps: list[Dependency], n: int, seed: int = 1) -> Summary:
    s = Sampler(random.Random(seed))
    return summarize([max(s.draw(d) for d in deps) for _ in range(n)])


def correlated_sequential(deps: list[Dependency], n: int, slow_probability: float, slow_factor: float, seed: int = 1) -> Summary:
    """All dependencies slow down together sometimes, as with a shared host or network problem."""
    rng = random.Random(seed)
    s = Sampler(rng)
    totals = []
    for _ in range(n):
        factor = slow_factor if rng.random() < slow_probability else 1.0
        totals.append(sum(s.draw(d) * factor for d in deps))
    return summarize(totals)


def with_deadline(deps: list[Dependency], budget_ms: float, n: int, seed: int = 1) -> dict[str, float]:
    """Sequential calls, each limited to what remains of the budget. Fails fast when none remains."""
    s = Sampler(random.Random(seed))
    ok = 0
    spent: list[float] = []
    for _ in range(n):
        remaining = budget_ms
        success = True
        for d in deps:
            t = s.draw(d)
            if t > remaining:
                remaining = 0
                success = False
                break
            remaining -= t
        spent.append(budget_ms - remaining)
        ok += success
    return {"success_pct": round(100 * ok / n, 2), "max_spent_ms": round(max(spent), 1)}


DEPS = [
    Dependency(name="auth", median_ms=10, sigma=0.4),
    Dependency(name="profile", median_ms=30, sigma=0.5),
    Dependency(name="pricing", median_ms=40, sigma=0.5),
    Dependency(name="inventory", median_ms=50, sigma=0.6),
]


def main() -> int:
    n = 100_000
    p99s = {d.name: round(own_p99(d, n, 7), 1) for d in DEPS}
    print(json.dumps({
        "per_dependency_p99_ms": p99s,
        "sum_of_p99s_ms": round(sum(p99s.values()), 1),
        "sequential_independent": sequential(DEPS, n).model_dump(),
        "sequential_correlated": correlated_sequential(DEPS, n, slow_probability=0.02, slow_factor=4).model_dump(),
        "parallel": parallel(DEPS, n).model_dump(),
        "deadline_300ms": with_deadline(DEPS, 300, n),
        "deadline_200ms": with_deadline(DEPS, 200, n),
    }, indent=2))
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

Terminal lab

Setup

cd topics/latency-budgets-and-critical-path-analysis/examples/typescript
pnpm install --ignore-workspace

Run

pnpm --silent start

Then run the simulation:

cd ../python && uv run python -m app.main | jq -c .

Test

cd ../typescript && pnpm test

cd ../python && uv run pytest -q

Expected output

Measured on the review date:

{"totalMs":265,"criticalPath":["gateway","auth","cart","fraud","payment","write-order"]}
{"id":"inventory","finish":100,"slack":30}
{"id":"pricing","finish":80,"slack":50}
{"optimizePricingTo10":{"before":265,"after":265,"savedMs":0}}
{"optimizeFraudTo40":{"before":265,"after":235,"savedMs":30}}

Pricing saves nothing, and a 50 ms improvement to fraud yields only 30 ms, as inventory takes over. The simulation prints:

{"per_dependency_p99_ms":{"auth":25.4,"profile":96.1,"pricing":128.2,"inventory":202.2},"sum_of_p99s_ms":451.9,"sequential_independent":{"p50_ms":141.3,"p99_ms":311.2},"sequential_correlated":{"p50_ms":142.7,"p99_ms":559.3},"parallel":{"p50_ms":63.4,"p99_ms":206.0},"deadline_300ms":{"success_pct":98.7,"max_spent_ms":300},"deadline_200ms":{"success_pct":86.0,"max_spent_ms":200}}

Notice that correlation barely changes p50, moving it from 141.3 to 142.7 ms, but it nearly doubles p99.

Cleanup

rm -rf ../typescript/node_modules .venv

Troubleshooting

• Cycle: errors name the loop in your graph, so remove one edge from it.
• Numbers differ slightly on a different Python version if the random generator's behavior changes.

Practical challenges

Industry standards and trends

Standards and specifications

W3C Trace Context, November 2021, standardizes trace propagation, which makes critical-path analysis across services possible. gRPC deadlines are part of that project's documentation, and they describe how deadlines propagate. HTTP has no standard header for deadlines: RFC 9110, June 2022, doesn't define one, so HTTP services use their own conventions.

Established practice

Dapper, Sigelman et al., 2010, shaped modern tracing. Timeouts, retries, and backoff with jitter, from the Amazon Builders' Library, discusses setting timeouts from measured latency. The Tail at Scale, 2013, is the standard reference on variability. These are published practices, not formal standards.

Managed-service options

Observability vendors offer trace waterfalls, and some calculate critical paths automatically. Service meshes can enforce per-route timeouts. Check whether a mesh's timeouts are coordinated with application retries.

Emerging approaches

Automated critical-path extraction from production traces is increasingly available in open-source and commercial tools. Adaptive deadlines and retries based on remaining budget are also used in some systems. Adoption data is limited, so treat this as an observation and not an established practice.

Recent changes

OpenTelemetry has become a common source for trace data across vendors, which lowers the cost of this kind of analysis. No formal standard change was identified at the review date.

Adoption constraints

This depends on trace coverage: a single uninstrumented hop appears as a gap. Budgets also require cross-team agreement. Sampling can miss rare slow paths, so keep slow traces preferentially.

Links in this section were checked on 2026-11-03 (access date). Items labelled as emerging or inferred are not settled facts and should be re-checked before you rely on them.

Interview preparation

Review

Key takeaways

• Sequential steps add, and parallel steps take the maximum.
• Only the critical path shortens a request, and savings are capped by the next-longest path.
• Slack tells you what not to optimize.
• Percentile budgets don't add: the sum may overstate or understate.
• Timeouts should come from the remaining budget, using propagated deadlines.
• Moving a step off the path often beats making it faster.

Decision checklist

• Is the call graph from traces, not memory?
• Is this step on the critical path for slow requests?
• What is the saving, given the next-longest path?
• Do timeouts fit within the budget, including retries?
• Does each budget have an owner?

Production-readiness checklist

• The journey has an end-to-end SLO.
• Deadlines propagate across services.
• Per-step budgets and trends are on a dashboard.
• External steps have timeouts and a degradation plan.
• Mandatory steps can't be skipped under time pressure.

Glossary

• Critical path: the longest route through a request's graph.
• Slack: how far a step can slow without delaying the total.
• Reserve: deliberately unallocated budget.
• Deadline: an absolute time limit that's passed along a request.
• Earliest finish: the soonest a step can be done, given its dependencies.

Quiz

Further study

Next, read Deadlines and cancellation propagation (not yet written) for deadline propagation, Timeouts (not yet written) for timeouts, and Tail latency for variability in fan-out. Synchronous versus asynchronous composition (not yet written) shows how to take a step out of the critical path with asynchronous design.

Progress

Progress is stored only in this browser. Export it from the curriculum page.