HTTP/2 Connection Pooling for Web Push at Scale

Web push has an unusual traffic shape: tens of thousands of tiny HTTPS POSTs, each under 4 KB, all aimed at a handful of hostnames. Almost none of the work is in the request itself. At that shape, the cost of getting to the push service dominates everything, and connection reuse is the single largest throughput lever in the dispatcher.

Quick answer. Open one HTTP/2 pool per endpoint authority, keep it warm, and size in-flight concurrency to connections × SETTINGS_MAX_CONCURRENT_STREAMS rather than to an arbitrary number. In Node.js that means an undici.Pool per origin with allowH2: true, a generous keepAliveTimeout, and a semaphore that matches the negotiated stream ceiling. Moving from per-request connections to a tuned HTTP/2 pool typically moves sustained throughput by more than an order of magnitude on the same hardware.

Why Connection Reuse Dominates

A single push dispatch is roughly 1.5 KB of ciphertext plus headers, and the push service answers with an empty 201. The useful work is a few hundred microseconds of I/O. Setting up a fresh TLS 1.3 connection to reach it costs one TCP round trip plus one TLS round trip — on a 25 ms path to fcm.googleapis.com, that is 50 ms of pure setup before a single byte of your payload moves. Under TLS 1.2 it is three round trips and closer to 75 ms.

Put differently: on a cold connection, more than 95% of the wall-clock time of a push dispatch is spent establishing a connection you are about to throw away. Reuse that connection across 10,000 messages and the amortised setup cost falls to five microseconds per message.

Header compression is a smaller but real second effect. A push request carries a VAPID Authorization header of 300–400 bytes that is byte-identical for every request to the same audience. HPACK indexes it after the first request, so subsequent requests send a one-byte reference instead. Across a million-message campaign that is several hundred megabytes of egress that never leaves the box.

One Pool per Authority

Pools are per-origin by definition — an HTTP/2 connection is bound to a single host, so fcm.googleapis.com and updates.push.services.mozilla.com can never share one. What is a choice is whether you let a generic global agent create pools implicitly, or whether you construct one explicitly per authority and own its settings.

Own them explicitly. Implicit pools inherit a single set of timeouts and connection counts, which means the conservative numbers Apple needs get applied to FCM, or the aggressive numbers FCM tolerates get applied to a service that will throttle you for them. Explicit pools also give you a place to hang per-provider metrics, and they map one-to-one onto the adapters described in Multi-Provider Push Gateway Design.

SETTINGS_MAX_CONCURRENT_STREAMS Is the Real Ceiling

Your MAX_CONCURRENCY constant is not the concurrency limit. During the HTTP/2 handshake the server sends a SETTINGS frame containing SETTINGS_MAX_CONCURRENT_STREAMS, and that value — commonly 100 to 250 for large push services — is the number of requests that can be genuinely in flight on one connection. Ask for more and the client does not error; it silently queues the surplus locally until a stream frees up.

That silent queuing is what makes over-provisioned concurrency so misleading. Requests appear to be in flight, your in-flight gauge reads 800, and measured latency climbs — but the extra 600 are sitting in a client-side queue, not on the wire. You have moved the bottleneck into your own process and made every latency percentile lie.

The correct sizing is simple: read the negotiated value, multiply by your connection count, and cap the semaphore at that product minus a small safety margin. Add connections, not stream requests, when you need more parallelism.

import { Client } from 'undici';

// Read the ceiling the server actually advertised for this origin.
export async function probeStreamCeiling(origin) {
  const client = new Client(origin, { allowH2: true });
  try {
    await client.request({ path: '/', method: 'HEAD' }).catch(() => {});
    // undici exposes the negotiated session on the H2 client internals;
    // fall back to the RFC 7540 default when it is not observable.
    const session = client[Symbol.for('undici.client.h2session')];
    const advertised = session?.remoteSettings?.maxConcurrentStreams;
    return Number.isFinite(advertised) ? advertised : 100;
  } finally {
    await client.close();
  }
}

export function sizeSemaphore(connections, streamCeiling, safetyMargin = 0.85) {
  return Math.max(1, Math.floor(connections * streamCeiling * safetyMargin));
}

If the ceiling is not observable in your runtime, measure it empirically instead: ramp concurrency and watch the point at which added concurrency stops adding throughput and starts adding only latency. That inflection is the ceiling, and the method is the same one used in parallel vs sequential push sending benchmarks.

Keep-Alive and Idle Timeouts

A pool only helps if the connection survives between bursts. Three timeouts decide that, and they have to agree with each other and with the server.

keepAliveTimeout is how long your client keeps an idle connection open. keepAliveMaxTimeout caps it regardless of what the server suggests via a Keep-Alive response header. keepAliveTimeoutThreshold shaves a margin off the server’s advertised value so you close first rather than racing a server-side close.

That race is the important one. If the server closes an idle connection at the same instant you dispatch on it, the request fails with ECONNRESET or UND_ERR_SOCKET — a spurious error that looks like a provider outage and can trip a circuit breaker during a completely healthy period. Always keep your idle timeout comfortably below the server’s, and always retry a first-byte failure on an idle connection exactly once without counting it as a provider failure.

Campaign traffic is bursty by nature: a send at 09:00, nothing until 14:00. With a 4-second idle timeout every campaign pays a full cold start. A background keep-warm — one HEAD or a TTL: 0 no-op per authority every 30 seconds — costs a few requests an hour and removes the cold-start penalty entirely.

Tuning undici in Node.js

undici is the right client here because it exposes the pool primitives directly rather than hiding them behind a global agent. Build one Pool per authority and hand it to the adapter.

import { Pool } from 'undici';

const PROFILES = {
  'fcm.googleapis.com':                { connections: 4, streams: 100 },
  'updates.push.services.mozilla.com': { connections: 2, streams: 128 },
  'web.push.apple.com':                { connections: 1, streams: 48 },
  'notify.windows.com':                { connections: 1, streams: 16 }
};

export function createPool(origin, profile) {
  return new Pool(origin, {
    allowH2: true,                    // negotiate h2 via ALPN, fall back to 1.1
    connections: profile.connections, // sockets held open to this origin
    pipelining: 1,                    // ignored under h2; keep 1 for the h1 fallback
    keepAliveTimeout: 30_000,         // hold idle sockets for 30 s
    keepAliveMaxTimeout: 600_000,     // never honour a server hint beyond 10 min
    keepAliveTimeoutThreshold: 2_000, // close 2 s before the server would
    headersTimeout: 10_000,           // abandon if no response headers in 10 s
    bodyTimeout: 10_000,
    connect: {
      // Session resumption turns a repeat handshake into a single round trip.
      maxCachedSessions: 128,
      timeout: 5_000
    }
  });
}

// One warm pool per authority, created once at boot and never per request.
export const pools = new Map(
  Object.entries(PROFILES).map(([host, profile]) => [
    host,
    createPool(`https://${host}`, profile)
  ])
);

// Cheap keep-warm so a 5-hour gap between campaigns does not cost a cold start.
export function startKeepWarm(intervalMs = 30_000) {
  const timer = setInterval(() => {
    for (const pool of pools.values()) {
      pool.request({ path: '/', method: 'HEAD' })
        .then((res) => res.body.dump())
        .catch(() => {});
    }
  }, intervalMs);
  timer.unref();
  return () => clearInterval(timer);
}

Two settings deserve emphasis. allowH2: true is opt-in — without it undici speaks HTTP/1.1 and every one of your connections carries one request at a time, which is the single most common reason a “pooled” dispatcher fails to scale. And connections is sockets, not requests: setting it to 200 in the hope of 200-way parallelism gives you 200 TLS handshakes and 200 idle sockets, when four connections at a 100-stream ceiling already provide 400-way concurrency. Pair these pools with the semaphore layer from Message Batching & Throughput Optimization so the pool is never asked for more than it can carry.

TLS Handshake Cost

The handshake is both network latency and CPU. A fresh TLS 1.3 connection costs one round trip and an ECDHE key exchange plus certificate chain verification — on the order of 1–2 ms of CPU per handshake on a modern core. At 2,000 new connections per second that is two to four full cores burned on cryptography that produced no notifications.

Session resumption removes most of it. TLS 1.3 tickets let a returning client complete in a single round trip with far less asymmetric work, and undici’s maxCachedSessions is what keeps those tickets around. It matters most exactly when pooling is imperfect — during a deploy, after a network blip, or when a push service recycles connections on its own schedule.

Certificate verification is the part people forget. Each new connection re-validates the chain unless the runtime caches it. Keeping four long-lived connections per authority means that work happens four times a day instead of four thousand times a minute.

Measuring the Difference

Instrument the pool, not just the request. Three numbers tell you whether pooling is actually working:

1. New connections per minute per authority. Should be near zero in steady state. A number that tracks your request rate means you are not reusing anything.
2. Sockets in TIME_WAIT. Run ss -tan state time-wait | wc -l during a send. Flat means reuse; climbing into the thousands means connection churn.
3. Time-to-first-byte split from total latency. A large gap between the two under load means requests are queuing on saturated streams, and the fix is more connections, not more concurrency.

Then run the same campaign under each configuration and compare sustained throughput.

Those numbers are illustrative of the shape of the gain, not a promise — your absolute values depend on round-trip time, payload size, and how much CPU your encryption stage leaves free. What is reliably reproducible is the ordering: keep-alive beats no keep-alive by several times, HTTP/2 beats keep-alive by several times again, and correct ceiling-matched sizing adds a final large fraction on top. The batch-window interaction with these numbers is measured in optimal batch size for web push throughput.

Gotchas and Edge Cases

• allowH2 defaults to off. A pool without it is an HTTP/1.1 pool. Assert the negotiated protocol in a startup check rather than assuming ALPN did what you wanted.
• GOAWAY is routine, not an outage. Push services recycle connections after a request or time budget. A well-behaved client drains in-flight streams and opens a replacement; it must not count the GOAWAY as a provider failure.
• More connections is not always better. Past the point where the service’s own per-client limits bite, extra connections earn 429 instead of throughput. Raise connections only while the rate-limit share stays flat, then stop.
• Idle-timeout races produce phantom errors. A socket closed by the server microseconds before you write shows up as ECONNRESET. Retry once, transparently, and keep it out of the breaker’s failure tally.
• Pools must be created at boot, never per request. A pool constructed inside a handler is a fresh connection every time with extra bookkeeping — strictly worse than no pool at all.

Related

• Multi-Provider Push Gateway Design — the adapter layer these per-authority pools plug into.
• FCM vs Mozilla Autopush endpoint differences — why the two busiest authorities need different pool profiles.
• Message Batching & Throughput Optimization — the semaphore that must be sized to the pool’s real ceiling.
• Parallel vs sequential push sending benchmarks — the ramp method for finding that ceiling empirically.

Back to Multi-Provider Push Gateway Design

FAQ