PWA Testing Blind Spots

Most PWA testing stops at the moment navigator.serviceWorker.register('/sw.js') resolves. The console prints a cheerful confirmation, Lighthouse awards a green checkmark, and the team moves on to test

The "Service Worker Registered" Console Log Is a Red Herring

Most PWA testing stops at the moment navigator.serviceWorker.register('/sw.js') resolves. The console prints a cheerful confirmation, Lighthouse awards a green checkmark, and the team moves on to testing the React components. This is a category error. Service workers exist in a state machine—*installing*, *installed*, *activating*, *activated*, and *redundant*—and the vast majority of production bugs occur in the transitions between these states, not in the registration promise.

Consider the standard Workbox 7.0 precache manifest. When you deploy version v2.4.1 of your app, the new service worker downloads, hashes the assets, and enters the installed state. However, it remains in *waiting* until every tab running the old v2.4.0 service worker closes. During this window— which can last indefinitely if a user never closes their laptop lid—your application is in split-brain mode: the HTML loads from the new deployment, but the CSS and JS are served by the stale service worker from CacheStorage. If your testing strategy only validates the "fresh install" scenario (empty cache, new registration), you miss the race condition where clients.claim() fires before the new cache is fully populated, serving 404s for chunked assets generated by Webpack 5's splitChunks optimization.

The fix isn't manual verification; it's autonomous lifecycle testing. Platforms like SUSA run 10 concurrent personas across device farms, each simulating the "tab left open for 3 days" scenario—forcing skipWaiting events, validating that activate handlers migrate IndexedDB schemas without nuking user data, and ensuring that the message event listener correctly broadcasts CONTROLLING changes to all windowClient instances before the DOM renders.

Cache-First Strategies Are Footguns in Disguise

Workbox's StaleWhileRevalidate strategy is the default recommendation for a reason: it balances speed with freshness. But "stale while revalidate" assumes your revalidation logic actually works. In Chrome 120 and Safari 17.2, a cached opaque response (status 0, type opaque) from a cross-origin CDN will never trigger a revalidation fetch if the original request lacked the Date header or if the CDN returns Cache-Control: immutable. Your users are pinned to a version of lodash.js from six months ago, and your analytics show zero errors because the resource loads successfully—it's just the wrong resource.

The real blind spot is the interaction between the Cache API and HTTP semantics. When you cache a navigation request using NetworkFirst, the service worker stores the full HTML response. If your CI pipeline deploys a new index.html that references main.a3f2.js instead of main.8b1c.js, but the service worker serves the cached HTML from 24 hours ago, the browser requests the old chunk which no longer exists on the server (404). Your error monitoring (Sentry, Rollbar) won't catch this because the 404 happens in a fetch event inside the service worker context, not in the main thread. You need to explicitly test for "cache pollution" scenarios:

// sw.js - The trap
self.addEventListener('fetch', (event) => {
  if (event.request.mode === 'navigate') {
    event.respondWith(
      caches.match(event.request).then((response) => {
        // If this returns the old index.html, you're broken
        return response || fetch(event.request);
      })
    );
  }
});

A robust test suite must verify that the HTML in the cache matches the current deployment's build hash. At SUSA, we validate this by intercepting the fetch event, checking the ETag or content-hash meta tag of the cached HTML against the live version, and flagging a "cache drift" violation if they diverge by more than one deployment cycle.

iOS Safari Is a Hostile Runtime Environment

While Chrome treats PWAs as first-class citizens, iOS Safari 16.x through 17.4 remains a study in passive-aggressive API restrictions. The 50MB storage cap per origin (down from 200MB in iOS 15) isn't just a recommendation; it's an eviction trigger that fires silently when the device enters Low Power Mode or when the user hasn't opened the app in 7 days. Your offline-first architecture works beautifully in testing, then fails catastrophically for a user on a cross-country flight because iOS purged the CacheStorage to save battery.

The "Add to Home Screen" (A2HS) flow on iOS creates a separate WKWebView instance with distinct storage partitions. If your testing only uses Safari's "Responsive Design Mode" or simulators, you miss the fact that standalone mode on iOS 17 ignores theme-color meta tags in favor of the manifest's background_color, but only if the manifest is fetched within 30 seconds of the A2HS prompt. Miss that window, and your app launches with a white splash screen on dark mode devices—a UX failure that automated screenshot comparison tools catch, but only if they're running on actual iPad hardware, not macOS simulators.

Furthermore, iOS Safari lacks support for the beforeinstallprompt event entirely. Your "Install App" button, which works flawlessly on Android Chrome 120, is dead weight on iOS. If your testing matrix doesn't include manual verification of the display-mode: standalone media query detection logic, you won't notice that your "Install to access offline mode" CTA is showing to users who already installed the PWA, because window.navigator.standalone (the iOS proprietary check) returns true while matchMedia('(display-mode: standalone)').matches returns false in certain iOS 16.4 builds.

The beforeinstallprompt Promise Chain That Never Resolves

On Android Chrome, the beforeinstallprompt event is a one-shot deal. If the user dismisses the prompt, the userChoice promise resolves with {outcome: 'dismissed'}, but the event will not fire again until the engagement heuristic resets (typically 90 days, or until the manifest changes). Most testing strategies verify that the prompt *can* be triggered, but not that the *deferred* pattern—storing the event for a custom "Install Now" button—handles the dismissal edge case correctly.

The bug manifests in state management. Teams often store the event in a React ref or global variable:

let deferredPrompt;

window.addEventListener('beforeinstallprompt', (e) => {
  e.preventDefault();
  deferredPrompt = e; // Stored for later
  showInstallButton();
});

installButton.addEventListener('click', async () => {
  if (!deferredPrompt) return;
  deferredPrompt.prompt();
  const { outcome } = await deferredPrompt.userChoice;
  if (outcome === 'accepted') {
    deferredPrompt = null; // Clean up
  }
});

The blind spot: if the browser decides the site is no longer installable (e.g., the manifest 404s due to a CDN hiccup), deferredPrompt.prompt() throws a DOMException with code "NotAllowedError", but the global variable remains set. Your UI shows the install button indefinitely, leading to a dead click. Worse, if the user clears site data via Chrome Settings, the service worker unregisters, but your app's state believes it's still installable because the beforeinstallprompt event fired in the previous session.

Testing this requires manipulating the chrome://flags #bypass-app-banner-engagement-checks and simulating the 90-day reset, or using DevTools' Application > Manifest > "Add to homescreen" trigger while throttling the network to offline to force the manifest fetch failure. Few CI pipelines include this level of browser instrumentation.

Background Sync Ghost Events

The Background Sync API (Chrome 49+, Safari 16.4+) and Periodic Background Sync (Chrome 80+) are notoriously difficult to validate because they execute when your application is technically "closed." The standard DevTools "Background services" > "Sync" button simulates a sync event, but it does so while the page is active. It does not test the actual resurrection flow: browser wakes up the service worker from a terminated state, fires the sync event, and potentially encounters an IndexedDB connection failure because the database version was upgraded while the app was backgrounded.

Consider a one-shot sync registered during a flight booking:

navigator.serviceWorker.ready.then((registration) => {
  registration.sync.register('submit-payment').catch((err) => {
    // This catches immediate registration failures only
    console.error('Sync registration failed', err);
  });
});

The blind spot is the sync event handler's execution context. If the user closes the tab immediately after clicking "Pay," the service worker has approximately 30 seconds (per Chromium's implementation) to complete the fetch. However, if your event.waitUntil() promise chain includes a fetch() that hits a 502 Gateway Timeout after 25 seconds, the sync will retry exponentially, but only while the browser remains open. If the user reboots their phone, the pending sync events may disappear entirely on some Android OEM skins (Samsung One UI 6.0 is particularly aggressive about killing background services).

Periodic Background Sync is even more opaque. The API requires the PWA to be installed, the site to have a engagement score > 0, and the user to have granted notification permissions. Testing the periodicsync event requires waiting for the browser's internal scheduler (typically 12-24 hours), or using chrome://inspect/#service-workers to manually trigger the event with a custom tag. Most teams ship code like this without ever verifying the minInterval compliance:

self.addEventListener('periodicsync', (event) => {
  if (event.tag === 'fetch-news') {
    event.waitUntil(fetchAndCacheNews());
  }
});

If fetchAndCacheNews() throws an unhandled rejection, the periodic sync may be disabled by the browser without notifying the user. You need autonomous agents that keep the PWA installed for days, monitoring the chrome://net-export logs to confirm that background fetches actually occur when the device transitions from WiFi to cellular at 3 AM.

Offline Mode Is Not Binary

The navigator.onLine API is a lie. It returns true if the network interface is up, regardless of whether the internet is reachable. Your "Offline" banner component, which listens to window.addEventListener('offline', ...), won't trigger when the user connects to a hotel WiFi that requires a captive portal login. The browser is "online," but every fetch returns a 302 redirect to http://192.168.1.1/login, which your service worker likely caches as the navigation response, trapping the user in a redirect loop.

The modern reality is "Lie-Fi": 2G connections with 100% packet loss, or 5G with DNS resolution failures. Testing with Chrome DevTools' "Offline" checkbox is insufficient. You need to simulate "Slow 3G" with 0kb/s download and verify that your NetworkFirst strategy falls back to cache within the timeout window (typically 3 seconds for perceived performance).

More insidious is the partial connectivity scenario. Your app makes a GraphQL query that succeeds, but the subsequent REST call for user assets fails with net::ERR_INTERNET_DISCONNECTED. If your state management library (Redux Toolkit, Apollo Client) doesn't handle this split-brain state—where data exists but assets don't—the UI renders broken image placeholders that are never retried because the service worker served a cached 404 for the image resource.

The File System Access API (showSaveFilePicker) compounds this. If a user initiates a save while offline, the promise rejects with an AbortError, but your app may have already optimistically updated the UI to show "Saved." Testing these flows requires network shapers like tc (Linux Traffic Control) on real devices, not just DevTools throttling, because DevTools doesn't accurately simulate the TCP handshake timeouts that occur on real cellular networks.

The Security Model Nobody Tests

Cross-Origin Isolation (COOP/COEP) breaks service workers in subtle ways. If your PWA enables Cross-Origin-Embedder-Policy: require-corp to use SharedArrayBuffer for WebAssembly threading (required by FFmpeg.wasm or Unity WebGL), every subresource—including those cached by the service worker—must either be same-origin or served with Cross-Origin-Resource-Policy: cross-origin.

The blind spot: opaque responses (from CDNs without CORS headers) cannot be read by the service worker, but they *can* be cached. However, when COEP is enabled, the browser refuses to load these cached opaque responses into the document, even if they were cached before the policy was applied. Your service worker serves a valid cached image, the browser rejects it, and the image appears broken. This only manifests in production builds with security headers enabled, not in localhost development where COEP is often disabled.

Content Security Policy (CSP) for service workers is another minefield. The script-src directive applies to the service worker script itself during installation, but connect-src applies to all fetch() calls made by the service worker. If your CSP changes to restrict API endpoints (e.g., moving from api.example.com to graphql.example.com), the service worker will fail to sync data, but the error won't bubble up to the main thread's window.onerror because service workers run in a separate execution context. You must explicitly test CSP violations in the service worker scope by monitoring securitypolicyviolation events in the worker context:

self.addEventListener('securitypolicyviolation', (e) => {
  // Report to analytics
  console.error('SW CSP violation:', e.blockedURI);
});

Additionally, Subresource Integrity (SRI) hashes in your HTML conflict with service worker precaching if the precache manifest doesn't strip the integrity attributes before hashing. Workbox 7.0 handles this correctly, but custom build pipelines often generate different hashes for the same file due to newline normalization, causing the browser to reject the cached resource despite it being byte-identical.

Cross-Browser DevTools Are Gaslighting You

Chrome DevTools' "Update on reload" checkbox (Application > Service Workers) is a trap. When enabled, it forces the service worker to skip the waiting phase and immediately activate, which masks lifecycle bugs that affect real users. Safari 17's Web Inspector, conversely, lacks a Service Worker panel entirely on iOS; you must connect an iPhone to a Mac and use the "Develop" menu, but even then, you cannot simulate "Offline" mode for service workers—they always bypass cache when the Mac host has connectivity.

Firefox 123 (Desktop) shows service workers in about:debugging, but the "Unregister" button is asynchronous and doesn't wait for activate event completion, leading to state leakage between tests. Edge cases like background_fetch events are only debuggable in Chrome, forcing teams to maintain separate test matrices for each browser's implementation of the FetchEvent lifecycle.

The Network tab in all browsers lies about who handled the request. If the service worker returns a cached response, the timing chart shows a network request with 0ms duration, but it won't show you that the cache match took 200ms because your CacheStorage contains 10,000 opaque responses and caches.match() is O(n). You need to instrument the service worker itself with performance marks:

self.addEventListener('fetch', (event) => {
  const start = performance.now();
  event.respondWith(
    caches.match(event.request).then((response) => {
      const duration = performance.now() - start;
      if (duration > 100) {
        console.warn(`Slow cache lookup: ${duration}ms for ${event.request.url}`);
      }
      return response || fetch(event.request);
    })
  );
});

Automated Regression for Ephemeral Service Workers

Traditional E2E frameworks (Cypress 13.x, Playwright 1.41) struggle with service worker testing because they run in a Node.js context with limited control over the browser's service worker lifecycle. Playwright's context.serviceWorkers() API returns the worker instance, but route interception happens in the Playwright context, not the service worker context, meaning you cannot test how the service worker handles a 503 error unless you control the server.

Cypress explicitly disables service workers by setting --disable-features=ServiceWorker in some versions unless chromeWebSecurity: false is set, which breaks CORS testing. Neither tool effectively simulates the "user has 50 tabs open and the OS terminates the service worker to reclaim memory" scenario, which is the primary cause of "Aw, Snap!" crashes in production PWAs.

This is where autonomous QA platforms close the gap. SUSA's CLI (npx susa test --pwa) deploys the build to a fleet of physical devices (Pixel 8 on Android 14, iPhone 15 on iOS 17.3) and executes "personas" that specifically target service worker fragility: installing the PWA, enabling airplane mode, triggering a background sync, killing the app via the OS app switcher, then reopening it to verify that the sync queue persisted. The platform generates Appium scripts that interact with the Android system UI to clear app data—simulating the 50MB iOS storage eviction—and verifies that the PWA gracefully degrades to a "Storage Full" warning rather than a white screen.

The output is a JUnit XML report that CI systems (GitHub Actions, GitLab CI) can consume, with specific failure modes like SW_LIFECYCLE_TIMEOUT or CACHE_DRIFT_DETECTED, rather than generic AssertionError: expected true to equal false.

The Update Flow That Deletes User Data

The most destructive blind spot is the activate event handler used for cache cleanup. The standard pattern purges old caches:

self.addEventListener('activate', (event) => {
  event.waitUntil(
    caches.keys().then((cacheNames) => {
      return Promise.all(
        cacheNames
          .filter((name) => name !== CURRENT_CACHE)
          .map((name) => caches.delete(name))
      );
    })
  );
});

This is safe for static assets, but if your service worker also manages an IndexedDB database (e.g., using Dexie.js 3.2 or idb 8.0), and you upgrade the DB schema in v2.0.0, the old service worker in v1.9.0 might still be writing to the old object stores when the new version activates. If the new version's onupgradeneeded handler calls db.deleteObjectStore('legacyStore') without checking if the old service worker has finished its pending writes, you lose user data.

The correct pattern requires coordination: the new service worker must broadcast a message to all old clients, wait for clients.claim() to take control, and only then migrate the database. Most teams skip this because it's hard to test—you need two versions of the app deployed simultaneously, and you need to force the browser to keep the old version alive in one tab while opening the new version in another.

Test this by opening https://app.example.com in Tab A (v1), then deploying v2 to the server, opening Tab B (which installs the waiting v2 worker), and verifying that Tab A receives the controllerchange event before any DB writes occur. Automated platforms like SUSA orchestrate this by maintaining persistent browser profiles across deployments, ensuring that the "waiting" phase doesn't become a "data loss" phase.

You cannot test service worker updates with a single page load. You must test the *delta* between deployments, treating the service worker as a stateful backend service that requires migration scripts, not a static file.