The performance effects of too much lazy loading

Data-driven advice for lazy loading images with Core Web Vitals in mind.

Felix Arntz

Rick Viscomi

Lazy loading is a technique that defers downloading a resource until it's needed, to conserve data and reduce network contention for critical assets. It became a web standard in 2019 and today loading="lazy" for images is supported by most major browsers.

This guide summarizes how publicly available web transparency data and ad hoc A/B testing was analyzed to understand the adoption and performance characteristics of browser-level image lazy loading. The findings included that lazy loading can be an amazingly effective tool for reducing unneeded image bytes, but overuse can negatively affect performance. Concretely, this analysis shows that more eagerly loading images within the initial viewport—while liberally lazy loading the rest—can give us the best of both worlds: fewer bytes loaded and improved Core Web Vitals.

Adoption

According to the most recent data in HTTP Archive, built-in image lazy loading is used by 29% of websites and adoption is growing rapidly.

Querying the raw data in the HTTP Archive project gives us a clearer understanding of what kinds of websites are driving adoption: 84% of sites that use browser-level image lazy loading use WordPress, 2% use another CMS, and the remaining 14% don't use a known CMS. These results make clear how WordPress is leading the charge in adoption.

The rate of adoption is also worth noting. One year ago in July 2020, WordPress sites that use lazy loading made up tens of thousands websites in the corpus of about 6 million (1% of total). Lazy loading adoption in WordPress alone has since grown to over 1 million websites (14% of total).

Correlational performance

Digging deeper into HTTP Archive, it can be compared how pages with and without browser-level image lazy loading perform with the Largest Contentful Paint (LCP) metric. The LCP data comes from real-user experiences from the Chrome User Experience Report (CrUX) as opposed to synthetic testing in the lab. The following chart uses a box-and-whisker plot to visualize the distributions of each pages' 75th percentile LCP: the lines represent the 10th and 90th percentiles and the boxes represent the 25th and 75th percentiles.

The median page without lazy loading has a 75th percentile LCP of 2,922 milliseconds, compared to 3,546 milliseconds for the median page with lazy loading. Overall, websites that use lazy loading tend to have worse LCP performance.

It's important to point out that these are correlational results and they don't necessarily point to lazy loading as being the cause of the slower performance. Hypothetically, if WordPress sites tend to be a bit slower, and given how much they make up the lazy loading cohort, that could explain the difference. To eliminate that variability, the focus can be narrowed down specifically to WordPress sites.

Unfortunately, the same pattern emerges when drilling down into WordPress pages; those that use lazy loading tend to have slower LCP performance. The median WordPress page without lazy loading has a 75th percentile LCP of 3,495 milliseconds, compared to 3,768 milliseconds for the median page with lazy loading.

This still doesn't prove that lazy loading causes pages to get slower, but using it does coincide with having slower performance. To try to answer the causality question, a lab-based A/B test was set up.

Causal performance

The goal for the A/B test was to prove or disprove the hypothesis that built-in image lazy loading, as implemented in WordPress core, resulted in slower LCP performance and fewer image bytes. The methodology used was to test a demo WordPress website with the twentytwentyone theme. Both the archive and single page types were tested, which are like the home and article pages, on desktop and emulated mobile devices using WebPageTest. Each combination of pages with and without lazy loading enabled were tested, and each test was ran nine times to get the median LCP value and number of image bytes.

These results compare the median LCP in milliseconds for tests on archive and single pages for desktop and mobile. When lazy loading was disabled on archive pages, LCP improved by a significant margin. On single pages, however, it made less of a difference.

Disabling lazy loading seems to make the single pages slightly faster. However, the difference in LCP is less than one standard deviation for both desktop and mobile tests, so this could be attributed to variance and consider the change neutral overall. By comparison, the difference for archive pages is closer to two to three standard deviations.

These results compare the median number of image bytes (in KB) for each test. As expected, lazy loading has a very clear positive effect on reducing the number of image bytes. If a real user were to scroll through the entire page, all images would load anyway as they cross into the viewport, but these results show the improved performance of the initial page load.

To summarize the results of the A/B test, the lazy loading technique used by WordPress very clearly helps reduce image bytes at the cost of a delayed LCP.

Testing a fix

The most important aspect of WordPress' current lazy-loading implementation for this experiment is that it lazy-loads images within the viewport (above the fold). The CMS blog post acknowledges this as a pattern to avoid, but experimental data at the time indicated that the effect on LCP was minimal and worth simplifying the implementation in WordPress core.

Given this new data, an experimental fix was created that avoids lazy loading images that are above the fold, and the fix was tested under the same conditions as the first A/B test.

These results are much more promising. Lazy loading only the images below the fold results in a complete reversal of the LCP regression and possibly even a slight improvement over disabling lazy loading entirely. How could it be faster than not lazy loading at all? One explanation is that by not loading below-the-fold images, there's less network contention with the LCP image, which enables it to load more quickly.

In terms of image bytes, the fix has absolutely no change as compared to the default behavior. This is great because that was one of the strengths of the current approach.

This fix comes with some caveats. WordPress determines which images to lazy load on the server side, which means it doesn't know anything about the user's viewport size or whether images initially load within it. So the fix uses heuristics about the images' relative location in the markup to guess whether it loads in the viewport. Specifically, if the image is the first featured image on the page or the first image in the main content, it's assumed to be above the fold, or close to it, and it won't be lazy-loaded.

Page-level conditions like the number of words in the heading or the amount of paragraph text early in the main content can affect whether the image is within the viewport. There are also user-level conditions that might affect the accuracy of the heuristics, especially the viewport size and the use of anchor links that change the scroll position of the page.

For those reasons, it's important to acknowledge that the fix is only calibrated to provide good performance in the general case, and fine-tuning might be needed to make these results applicable to all real-world scenarios.

Implementation (:#implementation)

Now that a better way to lazy-load images has been identified, all of the image savings and faster LCP performance, how can sites start using it? The highest priority change is to submit a patch to WordPress core to implement the experimental fix. The guidance in the Browser-level lazy-loading for CMSs blog post will also be updated to clarify the negative effects of above-the-fold lazy loading and how CMSs can use heuristics to avoid it.

Since these best practices are applicable to all web developers, it may also be worth flagging lazy loading antipatterns in tools like Lighthouse. Refer to the feature request on GitHub if you're interested to follow along with progress on that audit. Until then, one thing developers could do to find instances of LCP elements being lazy-loaded is to add more detailed logging to their field data.

new PerformanceObserver((list) => {
  const latestEntry = list.getEntries().at(-1);

  if (latestEntry?.element?.getAttribute('loading') == 'lazy') {
    console.warn('Warning: LCP element was lazy loaded', latestEntry);
  }
}).observe({type: 'largest-contentful-paint', buffered: true});

The preceding JavaScript snippet will evaluate the most recent LCP element and log a warning if it was lazy-loaded.

This also highlights a sharp edge of the lazy loading technique and the potential for API improvements at the platform level. For example, there's an open issue in Chromium to experiment with natively loading the first few images eagerly, similar to the fix, despite the loading attribute.

Conclusion

If your site uses browser-level image lazy loading, check how it's implemented and run A/B tests to better understand its performance costs. It may benefit from more eagerly loading images above the fold. If you have a WordPress site, there will hopefully be a patch landing in WordPress core soon. And if you're using another CMS, make sure they're aware of the potential performance issues described here.

Trying out relatively new web platform APIs can come with both risks and rewards—they're called cutting edge features for a reason. While we're starting to get a sense of the thorniness of browser-level image lazy loading, we're also seeing the upsides of how to use it to achieve better performance.

Photo by Frankie Lopez on Unsplash