How prefetching helped Terra increase ads click-through rate by 30% and speed up Largest Contentful Paint.

Prefetching resources speeds up page load times and improves business metrics.

Sep 1, 2023

Guilherme Moser de Souza

TwitterGitHub

Thierno Thiam

TwitterGitHub

Prefetching is a technique used to speed up page loading by downloading resources—or even entire pages—which are likely to be needed in the near future. Research has shown that faster load times result in higher conversion rates and better user experiences.

Terra is one of the largest content portals from Brazil, offering entertainment, news, and sports with more than 63 million unique visitors per month. We’ve collaborated with Terra’s engineering team to improve the loading time of articles by using prefetching techniques on certain sections of their website.

This case study describes Terra’s journey implementation which resulted in a 11% ads click-through rate (CTR) increase on mobile, 30% ads CTR on desktop, and 50% reduction in the Largest Contentful Paint (LCP) times.

Prefetching strategy #

Prefetching has been around for a while, but it is important to use it carefully as it consumes extra bandwidth for resources that are not immediately necessary. This technique should be applied thoughtfully to avoid unnecessary data usage. In the case of Terra, articles are prefetched if the following conditions are met:

• Visibility of links to prefetched articles: Terra used the Intersection Observer API to detect viewability of the section containing the articles that they wanted to prefetch.
• Favorable conditions for increased data usage: As mentioned previously, prefetching is a speculative performance improvement that consumes extra data, and that may not be a desirable outcome in every situation. To reduce the likelihood of wasting bandwidth, Terra uses the Network Information API along with the Device Memory API to determine whether to fetch the next article. Terra only fetches the next article when:
  • The connection speed is at least 3G and the device has at least 4GB of memory,
  • or if the device is running iOS.
• CPU idle: Finally, Terra checks if the CPU is idle and able to perform extra work by using requestIdleCallback, which takes a callback to be processed when the main thread is idle, or by a specific (optional) deadline—whichever comes first.

Adhering to these conditions ensures that Terra only fetches data when necessary, which saves bandwidth and battery life, and minimizes the impact of prefetches that end up going unused.

When these conditions are met, Terra prefetches the articles present in the sections: "Related Content" and "Recommended for you" highlighted in blue below.

Business Impact #

In order to measure the impact of this technique, Terra first launched this feature in the "Related content" section of the article page. A UTM code helped them to differentiate between prefetched and non-prefetched articles for comparison purposes. After two weeks of successful A/B testing, Terra decided to add the prefetching functionality to the "Recommended for you" section.

As a result of prefetching articles, an overall increase of ads metrics and a reduction of LCP and Time to First Byte (TTFB) times were observed:

Prefetching—when used with care—greatly improves page load time, increases ads metrics, and reduces LCP time.

Technical details #

Prefetching can be achieved through the use of resource hints such as rel=prefetch or rel=preload, via libraries such as quicklink or Guess.js, or using the newer Speculation Rules API. Terra has chosen to implement this by using the fetch API with a low priority in combination with an Intersection Observer instance. Terra made this choice as it allows them to support Safari, which doesn't yet support other prefetching methods like rel=prefetch or the Speculation Rules API, and a full-featured JavaScript library wasn't necessary for Terra's needs.

The below JavaScript is approximately equivalent to the code used by Terra:

function prefetch(nodeLists) {
  // Exclude slow ECTs < 3g
  if (navigator.connection &&
    (navigator.connection.effectiveType === 'slow-2g'
      || navigator.connection.effectiveType === '2g')
  ) {
    return;
  }

  // Exclude low end device which is device with memory <= 2GB
  if (navigator.deviceMemory && navigator.deviceMemory <= 2) {
    return;
  }

  const fetchLinkList = {};

  const observer = new IntersectionObserver(function (entries) {
    entries.forEach(function (entry) {
      if (entry.isIntersecting) {
        if (!fetchLinkList[entry.target.href]) {
          fetchLinkList[entry.target.href] = true;

          fetch(entry.target, {
            priority: 'low'
          });
        }

        observer.unobserve(entry = entry.target);
      }
    });
  });
}

const idleCallback = window.requestIdleCallback || function (cb) {
  let start = Date.now();

  return setTimeout(function () {
    cb({
      didTimeout: false,
      timeRemaining: function () {
        return Math.max(0, 50 - (Date.now() - start));
      }
    });
  }, 1);
}

idleCallback(function () {
  prefetch(nodeLists)
})

• The prefetch function first checks for a minimum connection quality and device memory before initiating prefetching.
• Then it uses an IntersectionObserver to monitor when elements become visible in the viewport, and subsequently adds URLs to a list for prefetching.
• The prefetch process is scheduled with requestIdleCallback, aiming to execute the prefetch function when the main thread is idle.

Conclusion #

When used with care, prefetching can significantly reduce load times for future navigation requests, thereby reducing friction in the user journey and increasing engagement. Prefetching results in loading of extra bytes that may not be used, so Terra took extra steps to only prefetch under good network conditions and on capable devices, where this information is available.

Special thanks to Gilberto Cocchi, Harry Theodoulou, Miguel Carlos Martínez Díaz, Barry Pollard, Jeremy Wagner, and Leonardo Bellini and Lucca Paradeda from Terra's Engineering team for their contribution to this work.