The performance effects of too much lazy loading

Data-driven advice for lazy loading images with Core Web Vitals in mind.

Lazy loading is a technique that defers downloading a resource until it's needed, to conserve data and reduce network contention for critical assets. It became a web standard in 2019 and today loading="lazy" for images is supported by most major browsers.

This page summarizes our analysis of publicly available web transparency data and ad hoc A/B testing to understand the adoption and performance characteristics of built-in image lazy loading. We found that lazy loading can be an amazingly effective tool for reducing unneeded image bytes, but overusing it can negatively affect site performance. Our analysis shows that eagerly loading images within the initial viewport, while lazy loading the rest, can give us the best of both worlds: fewer bytes loaded and improved Core Web Vitals.

Adoption

According to the most recent data in HTTP Archive, built-in image lazy loading is used by 29% of websites and adoption is growing rapidly.

Pie chart showing WordPress making up 84.1% of lazy loading adoption, other CMSs 2.3%, and non-CMSs 13.5%.
Breakdown of the types of websites that make use of built-in image lazy loading. (Source)

Querying the raw data in the HTTP Archive project gives us a clearer understanding of what kinds of websites are driving adoption: 84% of sites that use built-in image lazy loading use WordPress, 2% use another CMS, and the remaining 14% don't use a known CMS. These results make clear how WordPress is leading the charge in adoption.

Timeseries chart of lazy loading adoption with WordPress being the predominant player compared to other CMSs and non-CMSs, with similar proportions to the previous chart. Total adoption is shown to have rapidly increased from 1% to 17% from July 2020 to June 2021.
Breakdown of the types of websites that make use of built-in image lazy loading. (Source)

The rate of adoption is also worth noting. In July 2020, WordPress sites that use lazy loading made up tens of thousands websites in the corpus of about 6 million (1% of total). By the start of 2021, lazy loading adoption in WordPress alone had grown to over 1 million websites (14% of total).

Correlational performance

Digging deeper into HTTP Archive, we can compare how pages with and without built-in image lazy loading perform with the Largest Contentful Paint (LCP) metric. The LCP data comes from real-user experiences from the Chrome User Experience Report (CrUX) as opposed to synthetic testing in the lab. The following chart uses a box-and-whisker plot to visualize the distributions of each pages' 75th percentile LCP: the lines represent the 10th and 90th percentiles and the boxes represent the 25th and 75th percentiles.

Box and
    whisker chart showing the 10, 25, 75, and 90th percentiles for pages that do
    and do not use built-in image lazy loading. The LCP distribution of pages
    that don't use it is faster than those that do.
Distribution of all pages' 75th percentile LCP experience, broken down by whether they use built-in image lazy loading. (Source)

The median page without lazy loading has a 75th percentile LCP of 2,922 ms, compared to 3,546 ms for the median page with lazy loading. Overall, websites that use lazy loading tend to have worse LCP performance.

It's important to point out that these are correlational results,and they don't necessarily point to lazy loading as the cause of the slower performance. For example, if WordPress sites tend to be a bit slower, given how much of the lazy loading cohort they make up, that could explain the difference. So the next chart tries to eliminate that variability by looking only at WordPress sites.

Box and
    whisker chart showing the 10, 25, 75, and 90th percentiles for WordPress
    pages that do and do not use built-in image lazy loading. As in the previous
    chart, the LCP distribution of pages that don't use it is faster than those
    that do.
Distribution of WordPress pages' 75th percentile LCP experience, broken down by whether they use built-in image lazy loading. (Source)

Unfortunately, the same pattern emerges from only WordPress pages; those that use lazy loading tend to have slower LCP performance. The median WordPress page without lazy loading has a 75th percentile LCP of 3,495 ms, compared to 3,768 ms for the median page with lazy loading.

This still doesn't prove that lazy loading causes pages to load more slowly, but using it does coincide with having slower performance. To try to determine what causes this, we set up a lab-based A/B test.

Causal performance

The goal for the A/B test was to prove or disprove the hypothesis that built-in image lazy loading, as implemented in WordPress core, resulted in slower LCP performance and fewer image bytes. The methodology we used was to test a demo WordPress website with the twentytwentyone theme. We tested both archive and single page types, which are like the home and article pages, on desktop and emulated mobile devices using WebPageTest. We tested each combination of pages with and without lazy loading enabled and ran each test nine times to get the median LCP value and number of image bytes.

Series default disabled Difference from default
twentytwentyone-archive-desktop 2,029 1,759 -13%
twentytwentyone-archive-mobile 1,657 1,403 -15%
twentytwentyone-single-desktop 1,655 1,726 4%
twentytwentyone-single-mobile 1,352 1,384 2%
Change in LCP (ms) by disabling built-in image lazy loading on sample WordPress pages.

These results compare the median LCP in milliseconds for tests on archive and single pages for desktop and mobile. When we disabled lazy loading on archive pages, we saw LCP improve by a significant margin. On single pages, however, it made less of a difference.

Disabling lazy loading seems to make the single pages slightly faster. However, the difference in LCP is less than one standard deviation for both desktop and mobile tests, so we attribute this to variance and consider the change neutral overall. By comparison, the difference for archive pages is closer to two to three standard deviations.

Series default disabled Difference from default
twentytwentyone-archive-desktop 577 1173 103%
twentytwentyone-archive-mobile 172 378 120%
twentytwentyone-single-desktop 301 850 183%
twentytwentyone-single-mobile 114 378 233%
Change in the number of image bytes (KB) by disabling built-in image lazy loading on sample WordPress pages.

These results compare the median number of image bytes (in KB) for each test. As expected, lazy loading has a very clear positive effect on reducing the number of image bytes. If a real user were to scroll through the entire page, all images would load anyway as they cross into the viewport, but these results show the improved performance of the initial page load.

To summarize the results of the A/B test, the lazy loading technique used by WordPress very clearly helps reduce image bytes at the cost of a delayed LCP.

A potential fix

The most important aspect of WordPress' current lazy-loading implementation for this experiment is that it lazy-loads images within the viewport (_above the fold). The CMS blog post acknowledges this as a pattern to avoid, but experimental data at the time indicated that the effect on LCP was minimal and worth simplifying the implementation in WordPress core.

Given this new data, we created an experimental fix that avoids lazy loading images above the fold, and tested it under the same conditions as the first A/B test.

Series default disabled fix Difference from default Difference from disabled
twentytwentyone-archive-desktop 2,029 1,759 1,749 -14% -1%
twentytwentyone-archive-mobile 1,657 1,403 1,352 -18% -4%
twentytwentyone-single-desktop 1,655 1,726 1,676 1% -3%
twentytwentyone-single-mobile 1,352 1,384 1,342 -1% -3%
Change in LCP (ms) by the proposed fix for built-in image lazy loading on sample WordPress pages.

These results are much more promising. Lazy loading only the images below the fold results in a complete reversal of the LCP regression and possibly even a slight improvement over disabling LCP entirely. How could it be faster than not lazy loading at all? One explanation is that not loading below-the-fold images reduces network contention with the LCP image, which lets it load more quickly.

Series default disabled fix Difference from default Difference from disabled
twentytwentyone-archive-desktop 577 1173 577 0% -51%
twentytwentyone-archive-mobile 172 378 172 0% -54%
twentytwentyone-single-desktop 301 850 301 0% -65%
twentytwentyone-single-mobile 114 378 114 0% -70%
Change in the number of image bytes (KB) by the proposed fix for built-in image lazy loading on sample WordPress pages.

In terms of image bytes, the fix doesn't affect the default behavior. This is great, because that was one of the strengths of the default approach.

This fix comes with some caveats. WordPress determines which images to lazy load on the server side, which means it doesn't know anything about the user's viewport size or whether images initially load within it. So the fix uses heuristics about the images' relative location in the markup to guess whether it loads in the viewport. Specifically, if the image is the first featured image on the page or the first image in the main content, it's assumed to be above the fold, or close to it, and it won't be lazy-loaded. Page-level conditions like the number of words in the heading or the amount of paragraph text early in the main content can affect whether the image is within the viewport. There are also user-level conditions that might affect the accuracy of the heuristics, especially the viewport size and the use of anchor links that change the scroll position of the page. For those reasons, it's important to acknowledge that the fix is only calibrated to provide good performance in the general case, and fine-tuning might be needed to make these results applicable to all real-world scenarios.

Implementation (:#implement)

Now that we've identified a better way to lazy-load images, all of the data savings and faster LCP performance, how can we get sites to start using it? The highest-priority change is to submit a patch to WordPress core to implement the experimental fix. We'll also be updating the guidance in the Browser-level lazy-loading for CMSs blog post to clarify the negative effects of above-the-fold lazy loading and how CMSs can use heuristics to avoid it.

Since these best practices are applicable to all web developers, it might also be worth flagging lazy loading overuse in tools like Lighthouse. To follow the progress of that audit, refer to the feature request on GitHub. Until then, one thing developers can do to find instances of LCP elements being lazy loaded is to add more detailed logging to their field data.

The following code evaluates the most recent LCP element and logs a warning if it was lazy loaded.

new PerformanceObserver((list) => {
  const latestEntry = list.getEntries().at(-1);

  if (latestEntry?.element?.getAttribute('loading') == 'lazy') {
    console.warn('Warning: LCP element was lazy loaded', latestEntry);
  }
}).observe({type: 'largest-contentful-paint', buffered: true});

This also highlights a sharp edge of the lazy loading technique and the potential for API improvements at the platform level. For example, there's an open issue in Chromium to experiment with loading the first few images eagerly, similar to the fix, despite the loading attribute.

Conclusion

If your site uses built-in image lazy loading, check how it's implemented and run A/B tests to better understand its performance costs. It may benefit from more eagerly loading images above the fold. If you have a WordPress site, there will hopefully be a patch landing in WordPress core soon. If you're using another CMS, make sure they're aware of the potential performance issues described here.

Photo by Frankie Lopez on Unsplash