The Lie We Believe: Why Playwright's page.route Isn't Real Chaos Engineering

Your dashboard says 100% Pass. The CI pipeline is green. You deploy to production.

Five minutes later, support tickets start rolling in. Users on mobile networks are staring at a spinning wheel. An API timeout triggered a "White Screen of Death."

But wait... didn't we test for this? We used page.route() to simulate a 500 error. We used CDP to simulate "Slow 3G."

Here is the uncomfortable truth: Your tests lied to you.

Mocking network requests inside the browser is fast and convenient, but it creates a "happy bubble" that protects your code from the harsh reality of the internet.

Here is why browser-based mocking is failing you, and how to fix it.

The Leaky Abstraction

When you use page.route in Playwright, you are intercepting the request at the Application Layer, right inside the JavaScript engine.

You are effectively telling the browser: "If you see a request to /api/pay, just pretend it failed."

This is fine for checking if an error message appears. But it is useless for testing system resilience. Real network failures happen much lower, down in the dirty details of the TCP/IP stack.

page.route completely ignores the things that actually break production:

• DNS Failures: What happens if the domain name doesn't resolve? Your mock assumes the domain exists.
• SSL Handshakes: What if the security certificate expires? The browser kills the connection before page.route even sees it.
• Zombie Connections: Sometimes the server doesn't send a clean 500 error. It just accepts the connection and hangs. The TCP socket stays open, but no data flows. Browser mocks are terrible at simulating this specific type of "hanging" death.

You aren't testing how your app handles a broken network. You're testing how it handles a perfectly formatted JSON error.

The "Chrome-Only" Trap

Most QA engineers rely on Playwright’s network throttling to simulate "Offline" modes. Under the hood, this uses CDP (Chrome DevTools Protocol).

There is a major catch: CDP doesn't work in Safari (WebKit).

If 40% of your users are on iPhones, your chaos tests are ignoring them. You are optimizing your app for Chrome and hoping it works the same on Safari.

Spoiler: It doesn't. Safari handles network timeouts and retries very differently than Chrome. If you rely on CDP, you are flying blind on iOS.

The Solution: Get Out of the Browser

If we can't trust the browser to simulate a bad network, we have to break the network outside of the browser.

We need a Proxy.

Instead of mocking requests inside Playwright, we configure the browser to send all traffic through a "Chaos Proxy." This sits between your browser and the internet, manipulating real TCP packets.

Why this wins:

1. It’s Real: It drops packets, delays bytes, and cuts connections. No simulation.
2. It’s Agnostic: It works for Chrome, Firefox, Safari, and even mobile emulators.
3. It’s Full-Stack: It messes up requests from your frontend and your backend.

The Code

Talk is cheap. Let’s look at the difference in implementation.

1. The Trap (Fragile)

This is the standard approach using Chrome's internal protocol. Run this in Safari, and it will crash or silently fail.

def test_slow_network_chrome_only(page):
    # ❌ TRAP: This code relies on Chrome DevTools Protocol.
    # It will CRASH if you try to run it on WebKit (Safari).
    
    client = page.context.new_cdp_session(page)
    client.send("Network.emulateNetworkConditions", {
        "offline": False,
        "latency": 2000,  # Artificial 2s lag
        "downloadThroughput": 500 * 1024,
        "uploadThroughput": 500 * 1024,
    })

    page.goto("/dashboard")
    
    # If the network logic is client-side only, this might pass.
    # But DNS and SSL issues are completely ignored here.
    expect(page.locator(".skeleton-loader")).to_be_visible()

test('simulate slow network (chrome only)', async ({ page }) => {
  // ❌ TRAP: This relies on Chrome DevTools Protocol.
  // It fails or does nothing on WebKit (Safari) & Firefox.

  const client = await page.context().newCDPSession(page);
  await client.send('Network.emulateNetworkConditions', {
    offline: false,
    latency: 2000, // Artificial 2s lag
    downloadThroughput: 500 * 1024,
    uploadThroughput: 500 * 1024,
  });

  await page.goto('/dashboard');

  // We are testing the browser's internal logic, not the real network.
  await expect(page.locator('.skeleton-loader')).toBeVisible();
});

2. The Solution (Robust)

Here is the resilient way. We launch the browser (even WebKit!) and point it to a local proxy. The browser has no idea it's being manipulated. It just thinks the internet is terrible.

from playwright.sync_api import sync_playwright

def test_slow_network_real_proxy():
    with sync_playwright() as p:
        # ✅ SOLUTION: Works on WebKit, Firefox, and Chrome.
        # We point the browser to our Chaos Proxy (e.g. running on port 8080)
        browser = p.webkit.launch(
            proxy={
                "server": "http://127.0.0.1:8080"
                # The proxy is pre-configured to throttle traffic,
                # inject random TCP resets, or block DNS.
            }
        )
        
        page = browser.new_page()
        
        try:
            page.goto("http://my-app.com/dashboard")
        except Exception as e:
            # We catch REAL network errors here (e.g., Connection Reset)
            assert "Connection closed" in str(e)

        browser.close()

import { webkit } from 'playwright';

test('simulate slow network (real proxy)', async () => {
  // ✅ SOLUTION: Works on WebKit (Safari) too!
  // The browser has no idea it's being manipulated.
  
  const browser = await webkit.launch({
    proxy: {
      server: 'http://127.0.0.1:8080', 
      // All traffic (API, Assets, Third-party scripts) 
      // goes through the "Kill Zone".
    }
  });

  const page = await browser.newPage();
  
  try {
    await page.goto('http://my-app.com/dashboard');
  } catch (e) {
    // Asserting a real TCP failure
    expect(e.message).toContain('Network connection lost');
  }

  await browser.close();
});

Stop Mocking, Start Breaking

page.route is a fantastic tool for functional testing. Keep using it to check if your buttons work.

But if you want to verify Resilience—if you want to know if your app survives a subway tunnel or a backend crash—you need to leave the browser environment.

The problem with Proxies? They are painful to manage. Setting up Mitmproxy or Toxiproxy in a CI/CD pipeline usually means writing complex YAML configs and managing Docker containers. It’s "DevOps hell" just to delay a request.

That is why I am building Chaos Proxy. It’s a tool designed specifically for QA Engineers to inject real network chaos with a simple API, without the infrastructure headache.

Stop testing the Happy Path. Break your system before your users do.