Retry Logic & Backoff Strategies for Web Push

Automated retry mechanisms are the operational backbone of reliable web push delivery. Transient network partitions, browser service worker cold starts, and push gateway rate limits make synchronous fire-and-forget dispatches fundamentally unreliable. This guide details production-grade retry pipelines, backoff algorithms, and security controls required to maintain high delivery success rates while preserving endpoint trust and regulatory compliance.

Prerequisites

• A delivery state machine that emits classified outcomes (see Delivery Tracking & Acknowledgment)
• Redis (or an equivalent sorted-set / delayed-queue store) for scheduling retries
• A dead-letter queue for terminally failed payloads
• The original TTL value stored alongside each payload for window validation
• An HMAC secret for signing deferred retry tokens

1. Architectural Context & Retry Fundamentals

Within a robust Backend Delivery Architecture & Queue Management ecosystem, push delivery must be modeled as an asynchronous state machine rather than a linear HTTP call. Each payload transitions through deterministic states: QUEUED → DISPATCHING → AWAITING_ACK → RETRY or TERMINAL.

Delivery State Mapping & Retry Eligibility

Not all failures warrant retries. Implement a centralized status router to classify responses:

• Retryable (Transient): 502 Bad Gateway, 503 Service Unavailable, 504 Gateway Timeout, 429 Too Many Requests
• Terminal (Permanent): 400 Bad Request, 401 Unauthorized, 404 Not Found, 410 Gone (unsubscribed or expired endpoint)
• Inspect Before Retry: 413 Payload Too Large (requires payload size reduction before any retry)

Dead-Letter Queue (DLQ) Configuration

Permanently failed payloads must be routed to a DLQ to prevent infinite retry loops and resource exhaustion. Configure DLQ consumers to:

1. Archive payloads with immutable audit trails.
2. Trigger webhook alerts for SLA breach analysis.
3. Expose metrics for subscription hygiene reporting.

The DLQ is not a graveyard — it is the surface where you diagnose why a cohort of endpoints stopped accepting deliveries, and it needs its own schema, retention policy and replay path. Build it deliberately using the patterns in dead-letter queues for undeliverable push messages.

Implementation Directive: Establish baseline retry thresholds aligned with your SLA. For standard web push, cap retries at 3–5 attempts with a maximum wall-clock window of 24 hours. Beyond this threshold, the notification is stale and must be purged.

2. Exponential Backoff Algorithms & Jitter Implementation

Linear or fixed-interval retries cause thundering herd scenarios, overwhelming push gateways and triggering cascading 429 responses. Exponential backoff with randomized jitter distributes load probabilistically while preserving delivery urgency. A sustained 429 is its own discipline — the gateway is explicitly telling you to slow down via Retry-After, and ignoring it risks an IP-level ban; the dedicated response pattern is in Handling 429 Too Many Requests from push services.

Mathematical Formulation

delay_ms = min(MAX_DELAY, BASE_DELAY * 2^attempt) + random(-JITTER_RANGE, +JITTER_RANGE)

Where BASE_DELAY = 2000ms, MAX_DELAY = 60000ms, and JITTER_RANGE = 20% of the calculated base.

Production-Ready Implementation

The following TypeScript module demonstrates circuit-breaker-aware backoff scheduling. For a complete code walkthrough, reference Implementing exponential backoff for failed push deliveries.

import { Redis } from 'ioredis';

interface RetryConfig {
  baseDelayMs:   number;
  maxDelayMs:    number;
  maxAttempts:   number;
  jitterPercent: number;
}

interface RetryPayload {
  id:           string;
  endpoint:     string;
  vapidPublicKey: string;
  payload:      Buffer;
  attempt:      number;
  ttl:          number; // seconds
}

const CONFIG: RetryConfig = {
  baseDelayMs:   2000,
  maxDelayMs:    60000,
  maxAttempts:   5,
  jitterPercent: 0.2
};

export class PushRetryScheduler {
  private circuitOpenUntil: Map<string, number> = new Map();

  constructor(private redis: Redis) {}

  calculateBackoff(attempt: number): number {
    const exponential = Math.min(
      CONFIG.maxDelayMs,
      CONFIG.baseDelayMs * Math.pow(2, attempt)
    );
    const jitter = exponential * CONFIG.jitterPercent * (Math.random() * 2 - 1);
    return Math.max(0, Math.round(exponential + jitter));
  }

  async scheduleRetry(payload: RetryPayload): Promise<void> {
    if (payload.attempt >= CONFIG.maxAttempts) {
      await this.moveToDLQ(payload);
      return;
    }

    const region = this.extractRegion(payload.endpoint);
    if (this.isCircuitOpen(region)) {
      await this.deferToQueue(payload, this.circuitOpenUntil.get(region)!);
      return;
    }

    const delayMs = this.calculateBackoff(payload.attempt);
    const executionTime = Date.now() + delayMs;

    const key = `retry:${payload.id}:${payload.attempt}`;
    await this.redis.setex(
      key,
      Math.ceil(delayMs / 1000) + 60,
      JSON.stringify({ ...payload, scheduledAt: executionTime })
    );

    await this.redis.zadd('retry_queue', executionTime, key);
  }

  private isCircuitOpen(region: string): boolean {
    const openUntil = this.circuitOpenUntil.get(region);
    return openUntil ? Date.now() < openUntil : false;
  }

  private extractRegion(endpoint: string): string {
    return new URL(endpoint).hostname.split('.')[0] || 'default';
  }

  private async deferToQueue(payload: RetryPayload, until: number): Promise<void> {
    await this.redis.zadd('retry_circuit_delayed', until, JSON.stringify(payload));
  }

  private async moveToDLQ(payload: RetryPayload): Promise<void> {
    await this.redis.lpush('dlq:push', JSON.stringify({ ...payload, terminal: true }));
  }
}

3. TTL Alignment & Retry Window Constraints

Web push payloads carry a TTL header (RFC 8030) defining maximum gateway storage duration. Retrying beyond this window wastes compute and delivers stale content. Retry pipelines must synchronize with expiration parameters to enforce strict queue hygiene.

TTL-Aware Validation Logic

1. Parse & Store: Extract the TTL value from the initial dispatch request. Store it alongside retry metadata.
2. Pre-Retry Validation: Before dequeuing a retry, calculate remainingTTL = initialTTL - (currentTime - dispatchTime).
3. Auto-Purge: If remainingTTL <= 0, discard the payload and increment the TTL_EXPIRED_PURGE metric.

Coordinate this logic with TTL & Expiration Handling to maintain data freshness across distributed workers.

function isRetryWithinTTL(
  dispatchTimestamp: number,
  ttlSeconds: number,
  currentTime: number
): boolean {
  const elapsedSeconds = (currentTime - dispatchTimestamp) / 1000;
  return elapsedSeconds < ttlSeconds;
}

async function processRetryQueue(redis: Redis): Promise<void> {
  const now = Date.now();
  const items = await redis.zrangebyscore('retry_queue', 0, now, 'LIMIT', 0, 50);

  for (const key of items) {
    const raw = await redis.get(key);
    if (!raw) continue;

    const payload = JSON.parse(raw);
    if (!isRetryWithinTTL(payload.dispatchedAt, payload.ttl, now)) {
      await redis.del(key);
      // metrics.increment('ttl_expired_retry_purge');
      continue;
    }

    await dispatchPush(payload);
    await redis.del(key);
  }
}

4. Queue Prioritization & Batched Retry Execution

Unmanaged retries degrade system throughput and trigger gateway throttling. Implement priority-aware scheduling to ensure critical alerts bypass promotional noise.

Priority Tiering Strategy

• Tier 1 (Transactional): Password resets, 2FA codes, payment confirmations. Retry immediately with aggressive backoff.
• Tier 2 (Promotional): Campaigns, feature announcements. Standard backoff, batched dispatch.
• Tier 3 (Engagement): Re-engagement nudges, digest summaries. Lowest priority, deferred to off-peak windows.

Weighted Fair Queuing & Batch Alignment

Aggregate pending retries into time-sliced dispatch windows. Align concurrency limits with gateway rate limits. Integrate scheduling logic with Message Batching & Throughput Optimization to maximize delivery success without triggering throttling, and size each window against the concurrency ceiling established in optimal batch size for web push throughput.

interface RetryPayload {
  id:      string;
  attempt: number;
  tier?:   1 | 2 | 3;
  [key: string]: unknown;
}

function getTier(p: RetryPayload): number {
  return p.tier ?? 2;
}

function calculateJitter(count: number): number {
  return Math.random() * count * 10; // ms
}

interface PriorityBatch {
  tier:        number;
  payloads:    RetryPayload[];
  windowStart: number;
}

function prioritizeAndSlice(queue: RetryPayload[], maxBatchSize: number): PriorityBatch[] {
  const sorted = [...queue].sort((a, b) =>
    getTier(a) - getTier(b) || a.attempt - b.attempt
  );

  const batches: PriorityBatch[] = [];
  for (let i = 0; i < sorted.length; i += maxBatchSize) {
    const slice = sorted.slice(i, i + maxBatchSize);
    batches.push({
      tier: getTier(slice[0]),
      payloads: slice,
      windowStart: Date.now() + calculateJitter(slice.length)
    });
  }
  return batches;
}

5. Error Classification, Security & Compliance Alignment

Retry pipelines must enforce strict security boundaries and regulatory compliance. Deferred payloads are high-value targets for replay attacks and privacy violations if mishandled.

Secure Error Routing & Credential Handling

• Centralized Error Router: Map HTTP status codes to deterministic retry/abort decisions. Never retry on 4xx without explicit payload validation.
• HMAC-Signed Retry Tokens: Attach a time-bound HMAC signature to deferred payloads to prevent replay attacks. Verify signatures before dispatch.
• SSRF Prevention: Sanitize all endpoint URLs against RFC 3986 standards. Reject internal IPs, localhost, and non-HTTPS schemes before queuing.
• Encrypted State Storage: Store retry state in short-lived Redis keys with automatic expiration. Never persist VAPID keys or subscription endpoints in plaintext logs.

Regulatory Compliance Enforcement

• GDPR/CCPA: Implement real-time subscription sync. Halt retries immediately upon receiving an unsubscribe signal or 410 Gone response, and trigger the cleanup workflow in Handling 410 Gone responses at scale.
• Data Minimization: Redact endpoint URLs and VAPID keys from observability logs. Retain only anonymized delivery metrics and hashed subscription IDs.

Observability Metrics

Track the following KPIs to tune backoff parameters and identify gateway degradation:

Debugging & Troubleshooting Steps

1. Verify State Consistency: Cross-reference retry_queue Redis ZSET scores against worker execution timestamps. Desync indicates clock drift or network partition.
2. Inspect Circuit Breaker Logs: If retries stall, check regional gateway health endpoints. Manually reset circuitOpenUntil maps only after confirming upstream recovery.
3. Audit HMAC Failures: A spike in 401 Unauthorized during retries indicates VAPID key rotation or token expiration. Validate key lifecycle management.
4. Trace TTL Pruning: If TTL_EXPIRED_PURGE exceeds baseline, reduce initial TTL headers or tighten backoff multipliers for low-priority tiers.
5. Validate SSRF Filters: Test malformed endpoints in staging. Ensure URL parser rejects file://, ftp://, and private IP ranges before queue insertion.

6. Backoff Configuration Reference

These parameters define the shape of the backoff curve and the circuit breaker. Tighten them for transactional tiers and loosen them for promotional ones.

Verification

Confirm the curve and the TTL guard behave before relying on them. The delays should grow exponentially and never exceed maxDelayMs, and a payload past its window must be purged rather than dispatched:

import { strict as assert } from 'node:assert';

const scheduler = new PushRetryScheduler(redis);
const delays = [0, 1, 2, 3, 4].map((a) => scheduler.calculateBackoff(a));

// Monotonic-ish growth, capped, jitter-bounded
assert.ok(delays[4] <= 60000, 'capped at maxDelayMs');
assert.ok(delays.every((d) => d >= 0), 'no negative delays');

// A payload older than its TTL must be rejected before dispatch
const dispatchedAt = Date.now() - 4000 * 1000; // 4000s ago
assert.equal(isRetryWithinTTL(dispatchedAt, 3600, Date.now()), false);

In production, watch the observability metrics above: a rising TTL_EXPIRED_PURGE rate or frequent circuit trips signal that the curve is too slow for your TTLs or a gateway is degraded.

Sizing the Retry Budget

A backoff curve is only correct relative to two numbers you must choose deliberately: how much work a retry costs, and how long the message stays worth sending. Get either wrong and a well-implemented scheduler still produces the wrong behaviour.

Start from the cost side. Each attempt is one HTTP request over an already-warm connection plus one JWT signature, so the marginal CPU is small — but the marginal capacity is not, because retries share the same per-authority concurrency budget as first attempts. During a partial outage the retry stream can crowd out fresh sends entirely, which is the failure mode that turns a five-minute provider blip into an hour of backlog. Reserve capacity rather than hoping: give retries a separate, smaller slice of the concurrency budget for each authority, so a retry storm degrades retry latency instead of stalling new campaigns. The pooling mechanics that make that slice cheap are covered in HTTP/2 connection pooling for web push.

Then bound the curve by the message, not by a global constant. The useful ceiling on total retry duration is the remaining TTL, because an attempt made after the message expires is guaranteed waste even when the push service returns 201. Compute the remaining window at schedule time and drop the message when the next planned attempt would land outside it — the scheduler, not the worker, is the right place for that check, since it avoids waking a worker to discover the answer. A message with a 60-second TTL should get two fast attempts and then be abandoned; one with a 24-hour TTL can afford a curve stretching into hours. A single retry policy applied to both will over-retry the first and under-retry the second.

Two capacity numbers are worth writing down before an incident rather than during one. The first is the steady-state retry ratio: retried attempts divided by first attempts, which on a healthy pipeline sits low single digits and is dominated by 429s and transient 5xxs. The second is the drain time for a full backlog at your reserved retry concurrency — literally queue depth divided by reserved throughput. If that number exceeds the typical TTL of the messages in the queue, your backlog will expire rather than deliver, and no amount of retrying changes the outcome; the fix is more reserved concurrency or a shorter curve, not more attempts.

Alert on the ratio and the drain time, not on the raw retry count. Raw counts scale with campaign volume and will page you on a Tuesday morning send; the ratio isolates the thing that actually changed. Pair them with the dead-letter arrival rate described in dead-letter queues for undeliverable push messages, which is the signal that the curve has stopped absorbing failures and is now just deferring them.

Related

• Implementing exponential backoff for failed push deliveries — a full code walkthrough of the backoff scheduler.
• Handling 429 Too Many Requests from push services — the specific response pattern for rate-limit signals.
• Dead-letter queues for undeliverable push messages — schema, retention and replay design for payloads that exhaust their attempts.
• Delivery Tracking & Acknowledgment — the state machine that classifies which failures are retryable.
• TTL & Expiration Handling — the window that bounds how long a retry stays useful.

Back to Backend Delivery Architecture & Queue Management

FAQ