Message Batching & Throughput Optimization for Web Push Delivery

High-volume web push campaigns demand precise orchestration to manage provider throttling, minimize latency, and guarantee secure delivery. Implementing an efficient batching strategy is foundational to the broader Backend Delivery Architecture & Queue Management framework. This guide provides production-ready patterns for payload consolidation, connection pooling, and secure scaling.

Prerequisites

• An HTTP/2-capable push client (e.g. the web-push Node.js library) for connection reuse
• A broker-backed work queue (Redis, RabbitMQ, or Kafka) feeding the dispatch workers
• VAPID keys available at runtime via process.env.VAPID_PRIVATE_KEY
• Payloads pre-encrypted to aes128gcm and verified under the 4 KB ciphertext limit
• Metrics for in-flight request count, socket TIME_WAIT, and per-provider 429 rate

Architecting the Batching Engine

Push providers enforce strict rate limits and connection caps per origin. Unbatched, sequential dispatches exhaust connection pools and trigger 429 Too Many Requests responses. A deterministic batching engine groups subscriptions by push service, applies dynamic chunk sizing, and dispatches via a controlled worker pool.

Important: Web Push (RFC 8030) is a per-endpoint HTTP protocol — you send one HTTP POST per subscription. Batching in this context means controlling how many concurrent dispatches run simultaneously using a semaphore, not packaging multiple subscriptions into a single HTTP request.

Implementation Steps

1. Endpoint Normalization & Grouping: Partition subscription payloads by push service domain. Chrome/Edge endpoints route to FCM, Firefox to Mozilla Autopush, and Safari to APNs Web Push — each requires distinct Authorization header formats.
2. Dynamic Concurrency Limits: Configure a semaphore limiting concurrent in-flight requests to 10–20 per worker process. FCM handles high concurrency well; APNs benefits from more conservative limits to avoid HTTP/2 stream errors. Choosing the right window is a measurable trade-off rather than a guess — see Optimal batch size for web push throughput.
3. Priority Queue Dispatch: Deploy a worker pool that drains from a Redis-backed priority queue. High-priority campaigns (e.g., security alerts) bypass standard batching windows.

Production-Ready Dispatch Implementation

const BATCH_SIZE = 200;       // subscriptions per dispatch window
const MAX_CONCURRENCY = 10;   // concurrent HTTP requests per worker
const DISPATCH_TIMEOUT_MS = 30_000;

interface PushSubscription {
  endpoint: string;
  keys: { p256dh: string; auth: string };
  metadata?: Record<string, unknown>;
}

class Semaphore {
  private permits: number;
  private queue: (() => void)[] = [];

  constructor(permits: number) {
    this.permits = permits;
  }

  async run<T>(task: () => Promise<T>): Promise<T> {
    if (this.permits > 0) {
      this.permits--;
      try {
        return await task();
      } finally {
        this.permits++;
        this.releaseNext();
      }
    }
    return new Promise<T>((resolve, reject) => {
      this.queue.push(async () => {
        try { resolve(await task()); }
        catch (err) { reject(err); }
      });
    });
  }

  private releaseNext() {
    if (this.queue.length > 0) this.queue.shift()!();
  }
}

function chunkArray<T>(arr: T[], size: number): T[][] {
  const chunks: T[][] = [];
  for (let i = 0; i < arr.length; i += size) {
    chunks.push(arr.slice(i, i + size));
  }
  return chunks;
}

async function dispatchBatches(
  subscriptions: PushSubscription[],
  sendPush: (sub: PushSubscription) => Promise<void>
): Promise<void> {
  const semaphore = new Semaphore(MAX_CONCURRENCY);

  const dispatches = subscriptions.map((sub, index) =>
    semaphore.run(async () => {
      try {
        await Promise.race([
          sendPush(sub),
          new Promise<never>((_, reject) =>
            setTimeout(() => reject(new Error(`Dispatch ${index} timeout`)), DISPATCH_TIMEOUT_MS)
          )
        ]);
      } catch (error) {
        console.error(`[DISPATCH_ERROR] Sub ${index} failed:`, error);
        throw error;
      }
    })
  );

  await Promise.allSettled(dispatches);
}

Architecture Trade-offs: Higher concurrency increases throughput but risks TCP connection exhaustion and provider-side IP reputation degradation. Monitor TIME_WAIT socket counts to ensure they stay within OS limits.

Whether raising the semaphore actually buys throughput is worth measuring rather than assuming — Parallel vs sequential push sending benchmarks puts numbers on the gap between a strictly sequential loop and a bounded parallel dispatcher. The shape those measurements take is always the same: throughput climbs, plateaus, then falls as the provider starts refusing requests.

Optimizing Payload & Connection Throughput

Throughput bottlenecks frequently originate from redundant payload serialization and ephemeral HTTP connections. HTTP/2 multiplexing allows concurrent push requests over a single persistent TCP connection, drastically reducing TLS handshake overhead. Aligning expiration windows with TTL & Expiration Handling ensures stale payloads are discarded before consuming dispatch cycles.

Implementation Steps

1. HTTP/2 Persistence: Use an HTTP/2-capable client and reuse connections across requests to the same push service origin. The web-push Node.js library handles this internally.
2. Payload Minimization: Keep plaintext JSON payloads under 3 KB to stay within provider limits after encryption overhead. Send only IDs and action metadata; fetch full content client-side.
3. Pre-Encryption Pipeline: Encrypt payloads server-side before queue insertion. Offloading cryptographic operations to a dedicated pre-processing stage prevents CPU contention during high-throughput dispatch.

The cost being removed here is easy to underestimate. Without reuse, every single push message pays for a fresh TCP connection and a full TLS handshake to the same origin it just finished talking to.

Security & Compliance Posture

• PII Stripping: Remove or hash personally identifiable information before encryption. Push payloads traverse multiple network hops.
• VAPID Validation: Verify Authorization: WebPush <token> headers against active key rotation schedules. Invalid signatures result in immediate 401 Unauthorized responses.
• Regulatory Alignment: Enforce GDPR/CCPA consent flags at the batching layer. Subscriptions lacking explicit opt-in must be filtered before entering the dispatch pipeline.

Monitoring & Acknowledgment Integration

Without granular tracking, a single 410 Gone response within a dispatch window can corrupt subscription health metrics. Implementing a correlation ID per request and mapping it to individual subscription receipts enables accurate state reconciliation. This workflow directly feeds into Delivery Tracking & Acknowledgment systems, allowing real-time throughput adjustments and failure isolation.

Implementation Steps

1. Correlation Mapping: Attach a unique request_id (UUIDv4) and subscription_index to each HTTP dispatch. Store mappings in an ephemeral cache (e.g., Redis with 15-minute TTL).
2. Response Parsing: Parse 201 Created, 404 Not Found, and 410 Gone responses. Immediately flag 404/410 as invalid subscriptions and purge them from the active database.
3. Dead-Letter Routing: Route failed dispatches to a dedicated DLQ. Implement exponential backoff for transient 5xx errors as described in Retry Logic & Backoff Strategies, but permanently quarantine 4xx client errors to prevent retry storms.

Debugging Checklist

• Verify provider responses contain expected headers before abstracting status codes into internal metrics.
• Audit batch sizes against provider-specific Retry-After headers on 429 responses.
• Ensure worker logs capture raw HTTP status codes before internal mapping.

Production Deployment & Queue Scaling

Deploying a high-throughput dispatch system requires rigorous validation of connection limits, memory allocation during encryption, and circuit breaker thresholds. Align dispatch intervals with campaign velocity and enforce strict idempotency keys to prevent duplicate notifications. For infrastructure teams evaluating distributed message brokers, refer to Scaling push queues with Redis or RabbitMQ to select the optimal persistence and routing strategy.

Implementation Steps

1. VAPID Key Rotation Compatibility: Implement hot-swappable VAPID keys. Workers must fetch the latest key from a centralized secrets manager before signing each dispatch window.
2. Circuit Breaker Configuration: Set failure thresholds at 5% per provider. When exceeded, halt dispatch for 60 s, drain pending dispatches to a retry queue, and alert SRE teams.
3. Backpressure Handling: Enforce queue depth limits. When exceeded, apply producer-side rate limiting and return 503 Service Unavailable to upstream campaign APIs.
4. Load Testing Protocol: Simulate 50,000 concurrent subscription dispatches with a 99.5% success target. Monitor GC pauses, heap allocation, and network I/O saturation during peak windows.

The breaker is only useful if its transitions are explicit. Three states, four transitions, and one rule about what happens to in-flight work while the breaker is open.

Security & Operational Hardening

• Immutable Audit Logging: Record every dispatched request with correlation IDs, timestamps, and VAPID key fingerprints. Logs must be append-only and retained per compliance mandates.
• Idempotency Enforcement: Generate deterministic idempotency keys using SHA-256(campaign_id + subscription_endpoint_hash + timestamp_window). Reject duplicate dispatches within a 5-minute sliding window.
• Memory Safeguards: Cap worker heap usage at 70%. Implement stream-based payload processing to avoid loading entire subscription lists into memory simultaneously.

Worker Fleet Sizing

MAX_CONCURRENCY and worker count answer two different questions, and conflating them is the most common sizing mistake. In-flight requests per worker determine how much pressure a single process puts on one push-service origin; worker count determines how much total parallelism the fleet has. Raising the first pushes you toward 429; raising the second does not, as long as each worker keeps its own concurrency inside the provider’s tolerance. When a campaign is running late, add workers.

The per-worker rate is the only figure you cannot reason your way to. It follows directly from Little’s Law — sustained throughput equals in-flight requests divided by median round-trip time — but round-trip time depends on the provider, the region, the payload size and the TLS session state, so it has to be measured. Instrument the dispatcher to record the wall-clock time from request write to response head, take the median over a few thousand real sends per provider, and derive the rate from that. A worker holding sixteen in-flight requests at a 76 ms median sustains roughly 210 dispatches per second; the same worker against a provider with a 200 ms median sustains 80.

The headroom multiplier in step 4 is not padding. A first pass over a large audience typically produces a few per cent of 429 and 5xx responses, and each of those becomes a retry that competes with fresh traffic later in the window. If the fleet is sized exactly to the first pass, retries push the campaign past its deadline and, for time-bound content, past the point where the TTL makes delivery worthwhile.

Memory is the quiet constraint on a single worker. Each in-flight request holds its encrypted payload — up to 4 KB of ciphertext under the aes128gcm encoding — plus HTTP/2 frame buffers and the promise chain around it, so sixteen in-flight requests are trivial while sixteen hundred are not. Streaming subscriptions out of the queue in bounded pages, rather than materialising a campaign’s entire recipient list, is what keeps worker memory flat as the audience grows. The related failure is holding fully rendered payloads in the broker instead of references; that is a queue-sizing problem covered in Scaling push queues with Redis or RabbitMQ.

Throughput Tuning Reference

These are the knobs that move throughput without tipping a provider into throttling. Start conservative and raise concurrency only while the 429 rate stays near zero.

Verification

Validate the dispatcher under load before trusting it in a campaign. Watch the socket table and the provider response mix while ramping concurrency:

# Watch TIME_WAIT sockets while a load test runs against the dispatcher
watch -n 2 "ss -tan state time-wait | wc -l"

A healthy run holds TIME_WAIT well under the OS limit (HTTP/2 reuse keeps it low), shows a flat 429 rate, and reports a success ratio at or above your target (e.g. 99.5%). A climbing 429 rate means MAX_CONCURRENCY is too high for that provider — back it off and honor every Retry-After.

Re-measuring after a change

Treat the tuned concurrency as a measurement with an expiry date, not a constant. Three ordinary events invalidate it: a provider adjusting its own rate limits, a payload-size change that shifts the encryption cost per message, and a move to different instance sizes or a different egress path. Re-run the calibration whenever one of those lands, and record the resulting knee alongside the date and the payload profile you measured it with, so the next person can tell whether a regression is new or simply an old number applied to new conditions.

Keep the harness itself in the repository next to the dispatcher rather than in someone’s notes. A calibration you can re-run in ten minutes gets re-run; one that has to be rebuilt from memory does not, and the concurrency setting quietly becomes folklore. Where you run it against a real push service, keep the recipient set to endpoints you own — calibrating against live subscribers spends their attention on test traffic and will show up in your unsubscribe rate.

Related

• Scaling push queues with Redis or RabbitMQ — broker selection and persistence for the work queue behind these workers.
• Optimal batch size for web push throughput — how to measure the concurrency sweet spot per provider.
• Parallel vs sequential push sending benchmarks — measured throughput of a sequential loop against a bounded parallel dispatcher.
• Retry Logic & Backoff Strategies — backoff for the transient failures this dispatcher surfaces.
• Delivery Tracking & Acknowledgment — reconcile per-request correlation IDs into a delivery ledger.

Back to Backend Delivery Architecture & Queue Management

FAQ