How to Wait for Single Page Navigation and Re-hydration with Playwright and React Router

Jul 28, 2022

While testing this blog with the awesome Playwright I faced the problem of tests failing on CI server (github actions environment). After some trial-and-error and a bit of investigation I figured that all tests failing were initiating (or dependent on) partial application re-rendering or react rehydration.

Here I'd like to share my learning starting with the failing test and debugging process towards the reliable and universal approach to handling single page app navigation events in Playwright tests.

The Goal

website header we are testing

The test we will use as an example is the navigation bar (website header) test. We click a link there and confirm that navigation happened and target page showed up. Focus of the test is not to verify particular target page but to confirm that navigation bar links work as expected.

Locally the test was failing every other time so to increase the failure rate I turned the devTools network throttling on:

const cdpSession = await page.context().newCDPSession(page);

await cdpSession.send('Network.enable');

await cdpSession.send(
  'Network.emulateNetworkConditions', {
    downloadThroughput: 1000 * 1000, // 1Mb/s
    uploadThroughput: 512 * 1000,
    latency: 150,
    offline: false,
  },
);
typescript

Here is the test I started with:

await page.goto('https://lab.amalitsky.com');

// select the first post header
await page.waitForSelector('[data-testid=PostPreview-Header]');

// verify the header text
await expect(postHeader.innerText()).resolves.toContain('View and Component');
typescript

Here we open the home page with the list of Tech related posts and verify that the first post present on the page has the expected header.

ElementHandle.waitForSelector method has the auto-waiting functionality built into it. This is important because even when the page takes time to load and render our selector will patiently wait for the element to appear and become actionable. Default timeout is 30 seconds, I reduced it to 10 for all of my tests.

Now after we verified that we start testing on the home page, let's proceed with the actual navigation bar test.

We find the bar, click Cars link and validate the target Cars page after navigation. Cars page displays a different list of posts but structurally is the same as the home page with Tech posts.

await page.goto('https://lab.amalitsky.com');

const techPostHeader = await page.waitForSelector('[data-testid=PostPreview-Header]');

await expect(techPostHeader.innerText()).resolves.toContain('View and Component');

const navBar = await page.waitForSelector('[data-testid=Header]');

const carsPageLink = await navBar.waitForSelector('[data-testid*=HeaderTabs-Tab_cars]');

await carsPageLink.click();

const carsPostHeader = await page.waitForSelector('[data-testid=PostPreview-Header]');

await expect(carsPostHeader.innerText()).resolves.toContain('Audi B8');
typescript

The last assertion fails - it expects post header containing Audi B8 present on Cars page but receives the first home page post header instead.

Why is that?

Debugging

If we open captured playwright test trace we will see that the last assertion is run on the home page rather than on Cars page we were expecting.

Playwright Trace UI

Cars link is present in the navigation bar, is clickable and works every time in any environment when tested manually. It also uses the native Link component of the gatsbyJs library hence it is unlikely to be an implementation issue.

Documentation on elementHandle.click method states that it will wait for initiated navigations to either succeed or fail, unless noWaitAfter option is set.

But for some reason even though we await for the click method call to succeed, our assertion is executed too early while browser is still rendering the home page!

I guess from the browser (and subsequently, Playwright's) single page application navigation is not considered a browser-native navigation event hence it can't tell when it started or succeeded.

Also check this comment from the Playwright's contributor Dmitry Gozman where he is saying that Playwright waits only for "the navigation to be confirmed" and none of load/DOMContentLoadeded events.

Native Solution with Page Specific Selectors

The most obvious (and as obviously wrong) solution would be to wait for certain time and expect page transition and UI rendering to happen within that timeframe. Had to mention this option just so that I could formally knock it out.

The better and fairly common solution is to wait for some target page element to appear. Idea is that if some unique element has appeared it means that new application route got rendered, and we can proceed with our tests.

From implementation perspective this sounds pretty verbose since potentially we need to have some unique elements and selectors for every from-to page tuple.

Besides, using page content to verify successful navigation sounds like a leaking abstraction - state transition is an abstract event and doesn't really depend on from-to pages elements. I.e. in HTTP protocol using the HTTP response code is usually sufficient to understand if the request had failed or succeeded. We don't have to parse the HTTP response body to understand it.

If we update our selectors to be a little more specific our navbar test will never fail because of pending navigation and premature test execution.

await page.goto('https://lab.amalitsky.com');

// insted of selecting any post header we target very specific one with :has-test
// it also allows us to drop an extra assertion for the text itself
await page.waitForSelector('[data-testid=PostPreview-Header]:has-text("How to Wait")');

const navBar = await page.waitForSelector('[data-testid=Header]');

const carsPageLink = await navBar.waitForSelector('[data-testid*=HeaderTabs-Tab_cars]');

// initiating navigation
await carsPageLink.click();

// at this point we still might be on the home page
// but playwright will wait for the particular header to appear
await page.waitForSelector('[data-testid=PostPreview-Header]:has-text("Audi B8")');
typescript

This approach is reliable - our selectors are specific enough so that they don't overlap on different pages. If navigation happens within 10 seconds the test will find and match the expected element on the navigation target page.

It isn't as helpful when working around re-hydration delay since usually DOM tree structure doesn't change during re-hydration. There will be no new elements to wait for in order to capture the UI is ready state. Probably, you still can make it work with the addition of custom class or data-test-id attribute during re-hydration on a client.

React Router Location Change Event

Most single-page-applications use client-side router for seamless navigation between application routes. Such router navigations don't cause full page reload but update the view and browser's history.

Client side routers also provide methods to work with application route programmatically. In particular, they are able to announce (or otherwise communicate) application route change event via service method, react hook, or custom event.

Let's leverage the route change event to capture the successful single-page application navigation without the need to wait for certain page element to appear or disappear.

For this react driven website I'm using useLocation router hook wrapped in useEffect react hook.

That combo gives me the custom useLocationChangeEvent hook emitting specific CustomEvent on window object whenever application route changes:

import React, {
  useEffect,
} from 'react';

import { useLocation } from '@reach/router';

function useLocationChangeEvent(): void {
  const location = useLocation();

  useEffect(() => {
    if (window) {
      // custom event type provides location details as a payload
      const event = new CustomEvent('REACT_ROUTER_PAGE_CHANGE_EVENT', { detail: { location } });

      window.dispatchEvent(event);
    }
  }, [location]);
}
typescript

Now we need to capture REACT_ROUTER_PAGE_CHANGE_EVENT event in the Playwright test. So that once received we consider the application route changed and new page being ready for running subsequent tests.

For that we will use page.evaluate which allows us to wait for the window event in the browser context and resolve a promise in our test script environment once it was captured.

Basically we are building a bridge between react router useLocation hook and promise we can await for in the test.

Here is the wrapper implementation:

function waitForSpaNavigation(): Promise<string> {
  return new Promise((resolve, reject) => {
    let resolved = false;

    const eventListener = (event: Event): void => {
      // very lax check for TSC (typescript compiler)
      if (event instanceof CustomEvent) {
        resolved = true;

        // promise with be resolved with the URL pathname
        resolve((event.detail as unknown as any).location.pathname);
      }
    };

    window.addEventListener(
      'REACT_ROUTER_PAGE_CHANGE_EVENT',
      eventListener,
      { once: true }, // this is (at most) one time use event handler
    );

    // cleanup in case of timeout or non-event
    setTimeout(() => {
      window.removeEventListener(
        'REACT_ROUTER_PAGE_CHANGE_EVENT',
        eventListener,
      );

      if (!resolved) {
        reject(Error('Expected SPA navigation timeout'));
      }
    }, 10000); // timeout value better match playwright's timeout
  });
}
typescript

waitForSpaNavigation returns a Promise which will be resolved with navigation target page pathname or rejected if navigation didn't happen within the timeframe provided.

Under the hood it is setting a one time REACT_ROUTER_PAGE_CHANGE_EVENT event listener on window. Which is emitted by useLocationChangeEvent hook defined above on every route change.

It is important to execute waitForSpaNavigation page function before clicking the link or triggering navigation programmatically because otherwise REACT_ROUTER_PAGE_CHANGE_EVENT might get emitted before we get a chance to set up our listener.

That's how we use our custom navigation event approach in the test:

await page.goto('https://lab.amalitsky.com');

await page.waitForSelector('[data-testid=PostPreview-Header]:has-text("How to Wait")');

const navBar = await page.waitForSelector('[data-testid=Header]');

const carsPageLink = await navBar.waitForSelector('[data-testid*=HeaderTabs-Tab_cars]');

const [urlPath] = Promise.all([
  page.evaluate(waitForSpaNavigation),
  carsPageLink.click(),
]);

// now you can verify the succesfull navigation against the URL pathname
expect(urlPath).toMatchSnapshot();
typescript

Notice the Promise.all wrapper for click() and navigation event subscription - We don't want to await for the click before we set up the REACT_ROUTER_PAGE_CHANGE_EVENT event listener. Otherwise, we open ourselves to the potential race condition when navigation happens before we get a chance to catch our custom navigation event.

When using page.goto along with the custom navigation event listener don't forget to pass { waitUntil: 'commit' } option for the same reason:

// await is ok here since 'waitUntil: commit' resolves too early for the page to load
// and react router to instantiate and trigger our custom event
await page.goto('https://example.com', { waitUntil: 'commit' });

// now we wait for our custom "UI State Ready" event
await page.evaluate(waitForSpaNavigation);
typescript

This last snippet is particularly useful for SSR pages with client side re-hydration.

Conclusion

We figured that reliable multi-page Playwright tests in React/Vue/Angular application can be achieved with the use of page-specific selectors and web-first assertions or custom implementation of waitForSpaNavigation page function.

Unique selectors have the benefit of being straightforward and natural but there are cases where using they might feel too laborious or taxing.

And for those cases custom router navigation event which is accessible from the Playwright test offers a great alternative.

Thanks for reading this far! This post ended up being much longer than originally planned. Sorry about that and I hope it was helpful for you.

As always, please let me know what you think. Cheers!