The performance effects of too much lazy loading

Data-driven advice for lazy loading images with Core Web Vitals in mind.

Lazy loading is a technique to defer downloading a resource until it's needed, which conserves data and reduces network contention for critical assets. It became a web standard in 2019 and today loading="lazy" for images is supported by most major browsers. That sounds great, but is there such a thing as too much lazy loading?

This post summarizes how we analyzed publicly available web transparency data and ad hoc A/B testing to understand the adoption and performance characteristics of native image lazy loading. What we found is that lazy loading can be an amazingly effective tool for reducing unneeded image bytes, but overuse can negatively affect performance. Concretely, our analysis shows that more eagerly loading images within the initial viewport—while liberally lazy loading the rest—can give us the best of both worlds: fewer bytes loaded and improved Core Web Vitals.

Adoption

According to the most recent data in HTTP Archive, native image lazy loading is used by 17% of websites and adoption is growing rapidly. This much of a foothold in the ecosystem is remarkable for a relatively new API.

Pie chart showing WordPress making up 84.1% of lazy loading adoption, other CMSs 2.3%, and non-CMSs 13.5%.
Breakdown of the types of websites that make use of native image lazy loading. (Source)

Querying the raw data in the HTTP Archive project gives us a clearer understanding of what kinds of websites are driving adoption: 84% of sites that use native image lazy loading use WordPress, 2% use another CMS, and the remaining 14% don't use a known CMS. These results make clear how WordPress is leading the charge in adoption.

Timeseries chart of lazy loading adoption with WordPress being the predominant player compared to other CMSs and non-CMSs, with similar proportions to the previous chart. Total adoption is shown to have rapidly increased from 1% to 17% from July 2020 to June 2021.
Breakdown of the types of websites that make use of native image lazy loading. (Source)

The rate of adoption is also worth noting. One year ago in July 2020, WordPress sites that use lazy loading made up tens of thousands websites in the corpus of about 6 million (1% of total). Lazy loading adoption in WordPress alone has since grown to over 1 million websites (14% of total).

Correlational performance

Digging deeper into HTTP Archive, we can compare how pages with and without native image lazy loading perform with the Largest Contentful Paint (LCP) metric. The LCP data comes from real-user experiences from the Chrome User Experience Report (CrUX) as opposed to synthetic testing in the lab. The chart below uses a box-and-whisker plot to visualize the distributions of each pages' 75th percentile LCP: the lines represent the 10th and 90th percentiles and the boxes represent the 25th and 75th percentiles.

Box and whisker chart showing the 10, 25, 75, and 90th percentiles for pages that do and do not use native image lazy loading. Comparatively, the LCP distribution of pages that do not use it is faster than those that do.
Distribution of all pages' 75th percentile LCP experience, broken down by whether they use native image lazy loading. (Source)

The median page without lazy loading has a 75th percentile LCP of 2,922 ms, compared to 3,546 ms for the median page with lazy loading. Overall, websites that use lazy loading tend to have worse LCP performance.

It's important to point out that these are correlational results and they don't necessarily point to lazy loading as being the cause of the slower performance. Hypothetically, if WordPress sites tend to be a bit slower, and given how much they make up the lazy loading cohort, that could explain the difference. So let's try to eliminate that variability by looking only at WordPress sites.

Box and whisker chart showing the 10, 25, 75, and 90th percentiles for WordPress pages that do and do not use native image lazy loading. Comparatively, the LCP distribution of pages that do not use it is faster than those that do, similar to the previous chart.
Distribution of WordPress pages' 75th percentile LCP experience, broken down by whether they use native image lazy loading. (Source)

Unfortunately, the same pattern emerges when we drill down into WordPress pages; those that use lazy loading tend to have slower LCP performance. The median WordPress page without lazy loading has a 75th percentile LCP of 3,495 ms, compared to 3,768 ms for the median page with lazy loading.

This still doesn't prove that lazy loading causes pages to get slower, but using it does coincide with having slower performance. To try to answer the causality question, we set up a lab-based A/B test.

Causal performance

The goal for the A/B test was to prove or disprove the hypothesis that native image lazy loading, as implemented in WordPress core, resulted in slower LCP performance and fewer image bytes. The methodology we used was to test a demo WordPress website with the twentytwentyone theme. We tested both archive and single page types, which are like the home and article pages, on desktop and emulated mobile devices using WebPageTest. We tested each combination of pages with and without lazy loading enabled and ran each test nine times to get the median LCP value and number of image bytes.

Series default disabled Difference from default
twentytwentyone-archive-desktop 2,029 1,759 -13%
twentytwentyone-archive-mobile 1,657 1,403 -15%
twentytwentyone-single-desktop 1,655 1,726 4%
twentytwentyone-single-mobile 1,352 1,384 2%
Change in LCP (ms) by disabling native image lazy loading on sample WordPress pages.

The results above compare the median LCP in milliseconds for tests on archive and single pages for desktop and mobile. When we disabled lazy loading on archive pages, we observed LCP improving by a significant margin. On single pages, however, the difference was more neutral.

It's worth noting that the effect of disabling lazy loading actually appears to make the single pages slightly faster. However, the difference in LCP is less than one standard deviation for both desktop and mobile tests, so we attribute this to variance and consider the change neutral overall. By comparison, the difference for archive pages is more like two to three standard deviations.

Series default disabled Difference from default
twentytwentyone-archive-desktop 577 1173 103%
twentytwentyone-archive-mobile 172 378 120%
twentytwentyone-single-desktop 301 850 183%
twentytwentyone-single-mobile 114 378 233%
Change in the number of image bytes (KB) by disabling native image lazy loading on sample WordPress pages.

The results above compare the median number of image bytes (in KB) for each test. As expected, lazy loading has a very clear positive effect on reducing the number of image bytes. If a real user were to scroll the entire page down, all images would load anyway as they cross into the viewport, but these results show the improved performance of the initial page load.

To summarize the results of the A/B test, the lazy loading technique used by WordPress very clearly helps reduce image bytes but at the cost of a delayed LCP.

Testing a fix

Before we get into how the fix was implemented, let's look at how lazy loading works in WordPress today. The most important aspect of the current implementation is that it lazy-loads images above the fold (within the viewport). The CMS blog post acknowledges this as a pattern to avoid, but experimental data at the time indicated that the effect on LCP was minimal and worth simplifying the implementation in WordPress core.

Given this new data, we created an experimental fix that avoids lazy loading images that are above the fold and we tested it under the same conditions as the first A/B test.

Series default disabled fix Difference from default Difference from disabled
twentytwentyone-archive-desktop 2,029 1,759 1,749 -14% -1%
twentytwentyone-archive-mobile 1,657 1,403 1,352 -18% -4%
twentytwentyone-single-desktop 1,655 1,726 1,676 1% -3%
twentytwentyone-single-mobile 1,352 1,384 1,342 -1% -3%
Change in LCP (ms) by the proposed fix for native image lazy loading on sample WordPress pages.

These results are much more promising. Lazy loading only the images below the fold results in a complete reversal of the LCP regression and possibly even a slight improvement over disabling LCP entirely. How could it be faster than not lazy loading at all? One explanation is that by not loading below-the-fold images, there's less network contention with the LCP image, which enables it to load more quickly.

Series default disabled fix Difference from default Difference from disabled
twentytwentyone-archive-desktop 577 1173 577 0% -51%
twentytwentyone-archive-mobile 172 378 172 0% -54%
twentytwentyone-single-desktop 301 850 301 0% -65%
twentytwentyone-single-mobile 114 378 114 0% -70%
Change in the number of image bytes (KB) by the proposed fix for native image lazy loading on sample WordPress pages.

In terms of image bytes, the fix has absolutely no change as compared to the default behavior. This is great because that was one of the strengths of the current approach.

There are some caveats with this fix. WordPress determines which images to lazy-load on the server-side, which means that it doesn't know anything about the user's viewport size or whether images will initially load within it. So the fix uses heuristics about the images' relative location in the markup to guess whether it will be in the viewport. Specifically, if the image is the first featured image on the page or the first image in the main content, it's assumed to be above the fold (or close to it), and it will not be lazy-loaded. Page-level conditions like the number of words in the heading or the amount of paragraph text early in the main content may affect whether the image is within the viewport. There are also user-level conditions that may affect the accuracy of the heuristics, especially the viewport size and the usage of anchor links that change the scroll position of the page. For those reasons, it's important to acknowledge that the fix is only calibrated to provide good performance in the general case and fine-tuning may be needed to make these results applicable to all real-world scenarios.

Rolling it out

Now that we've identified a better way to lazy-load images, all of the image savings and faster LCP performance, how can we get sites to start using it? The highest priority change is to submit a patch to WordPress core to implement the experimental fix. We'll also be updating the guidance in the Browser-level lazy-loading for CMSs blog post to clarify the negative effects of above-the-fold lazy loading and how CMSs can use heuristics to avoid it.

Since these best practices are applicable to all web developers, it may also be worth flagging lazy loading antipatterns in tools like Lighthouse. Refer to the feature request on GitHub if you're interested to follow along with progress on that audit. Until then, one thing developers could do to find instances of LCP elements being lazy-loaded is to add more detailed logging to their field data.

new PerformanceObserver((list) => {
  const latestEntry = list.getEntries().at(-1);

  if (latestEntry?.element?.getAttribute('loading') == 'lazy') {
    console.warn('Warning: LCP element was lazy loaded', latestEntry);
  }
}).observe({type: 'largest-contentful-paint', buffered: true});

The JavaScript snippet above will evaluate the most recent LCP element and log a warning if it was lazy-loaded.

This also highlights a sharp edge of the lazy loading technique and the potential for API improvements at the platform level. For example, there's an open issue in Chromium to experiment with natively loading the first few images eagerly, similar to the fix, despite the loading attribute.

Wrapping it up

If your site uses native image lazy loading, check how it's implemented and run A/B tests to better understand its performance costs. It may benefit from more eagerly loading images above the fold. If you have a WordPress site, there will hopefully be a patch landing in WordPress core soon. And if you're using another CMS, make sure they're aware of the potential performance issues described here.

Trying out relatively new web platform APIs can come with both risks and rewards—they're called cutting edge features for a reason. While we're starting to get a sense of the thorniness of native image lazy loading, we're also seeing the upsides of how to use it to achieve better performance.

Photo by Frankie Lopez on Unsplash