The performance effects of too much lazy loading

Data-driven advice for lazy loading images with Core Web Vitals in mind.

Jul 15, 2021 — Updated Jul 18, 2022

Rick Viscomi

TwitterGitHubHomepage

Felix Arntz

TwitterHomepage

Lazy loading is a technique to defer downloading a resource until it's needed, which conserves data and reduces network contention for critical assets. It became a web standard in 2019 and today loading="lazy" for images is supported by most major browsers. That sounds great, but is there such a thing as too much lazy loading?

This post summarizes how we analyzed publicly available web transparency data and ad hoc A/B testing to understand the adoption and performance characteristics of native image lazy loading. What we found is that lazy loading can be an amazingly effective tool for reducing unneeded image bytes, but overuse can negatively affect performance. Concretely, our analysis shows that more eagerly loading images within the initial viewport—while liberally lazy loading the rest—can give us the best of both worlds: fewer bytes loaded and improved Core Web Vitals.

Adoption #

According to the most recent data in HTTP Archive, native image lazy loading is used by 17% of websites and adoption is growing rapidly. This much of a foothold in the ecosystem is remarkable for a relatively new API.

Querying the raw data in the HTTP Archive project gives us a clearer understanding of what kinds of websites are driving adoption: 84% of sites that use native image lazy loading use WordPress, 2% use another CMS, and the remaining 14% don't use a known CMS. These results make clear how WordPress is leading the charge in adoption.

The rate of adoption is also worth noting. One year ago in July 2020, WordPress sites that use lazy loading made up tens of thousands websites in the corpus of about 6 million (1% of total). Lazy loading adoption in WordPress alone has since grown to over 1 million websites (14% of total).

Correlational performance #

Digging deeper into HTTP Archive, we can compare how pages with and without native image lazy loading perform with the Largest Contentful Paint (LCP) metric. The LCP data comes from real-user experiences from the Chrome User Experience Report (CrUX) as opposed to synthetic testing in the lab. The chart below uses a box-and-whisker plot to visualize the distributions of each pages' 75th percentile LCP: the lines represent the 10th and 90th percentiles and the boxes represent the 25th and 75th percentiles.

The median page without lazy loading has a 75th percentile LCP of 2,922 ms, compared to 3,546 ms for the median page with lazy loading. Overall, websites that use lazy loading tend to have worse LCP performance.

It's important to point out that these are correlational results and they don't necessarily point to lazy loading as being the cause of the slower performance. Hypothetically, if WordPress sites tend to be a bit slower, and given how much they make up the lazy loading cohort, that could explain the difference. So let's try to eliminate that variability by looking only at WordPress sites.

Unfortunately, the same pattern emerges when we drill down into WordPress pages; those that use lazy loading tend to have slower LCP performance. The median WordPress page without lazy loading has a 75th percentile LCP of 3,495 ms, compared to 3,768 ms for the median page with lazy loading.

This still doesn't prove that lazy loading causes pages to get slower, but using it does coincide with having slower performance. To try to answer the causality question, we set up a lab-based A/B test.

Causal performance #

The goal for the A/B test was to prove or disprove the hypothesis that native image lazy loading, as implemented in WordPress core, resulted in slower LCP performance and fewer image bytes. The methodology we used was to test a demo WordPress website with the twentytwentyone theme. We tested both archive and single page types, which are like the home and article pages, on desktop and emulated mobile devices using WebPageTest. We tested each combination of pages with and without lazy loading enabled and ran each test nine times to get the median LCP value and number of image bytes.

The results above compare the median LCP in milliseconds for tests on archive and single pages for desktop and mobile. When we disabled lazy loading on archive pages, we observed LCP improving by a significant margin. On single pages, however, the difference was more neutral.

It's worth noting that the effect of disabling lazy loading actually appears to make the single pages slightly faster. However, the difference in LCP is less than one standard deviation for both desktop and mobile tests, so we attribute this to variance and consider the change neutral overall. By comparison, the difference for archive pages is more like two to three standard deviations.

The results above compare the median number of image bytes (in KB) for each test. As expected, lazy loading has a very clear positive effect on reducing the number of image bytes. If a real user were to scroll the entire page down, all images would load anyway as they cross into the viewport, but these results show the improved performance of the initial page load.

To summarize the results of the A/B test, the lazy loading technique used by WordPress very clearly helps reduce image bytes but at the cost of a delayed LCP.

Testing a fix #

Before we get into how the fix was implemented, let's look at how lazy loading works in WordPress today. The most important aspect of the current implementation is that it lazy-loads images above the fold (within the viewport). The CMS blog post acknowledges this as a pattern to avoid, but experimental data at the time indicated that the effect on LCP was minimal and worth simplifying the implementation in WordPress core.

Given this new data, we created an experimental fix that avoids lazy loading images that are above the fold and we tested it under the same conditions as the first A/B test.

These results are much more promising. Lazy loading only the images below the fold results in a complete reversal of the LCP regression and possibly even a slight improvement over disabling LCP entirely. How could it be faster than not lazy loading at all? One explanation is that by not loading below-the-fold images, there's less network contention with the LCP image, which enables it to load more quickly.

In terms of image bytes, the fix has absolutely no change as compared to the default behavior. This is great because that was one of the strengths of the current approach.

There are some caveats with this fix. WordPress determines which images to lazy-load on the server-side, which means that it doesn't know anything about the user's viewport size or whether images will initially load within it. So the fix uses heuristics about the images' relative location in the markup to guess whether it will be in the viewport. Specifically, if the image is the first featured image on the page or the first image in the main content, it's assumed to be above the fold (or close to it), and it will not be lazy-loaded. Page-level conditions like the number of words in the heading or the amount of paragraph text early in the main content may affect whether the image is within the viewport. There are also user-level conditions that may affect the accuracy of the heuristics, especially the viewport size and the usage of anchor links that change the scroll position of the page. For those reasons, it's important to acknowledge that the fix is only calibrated to provide good performance in the general case and fine-tuning may be needed to make these results applicable to all real-world scenarios.

Rolling it out #

Now that we've identified a better way to lazy-load images, all of the image savings and faster LCP performance, how can we get sites to start using it? The highest priority change is to submit a patch to WordPress core to implement the experimental fix. We'll also be updating the guidance in the Browser-level lazy-loading for CMSs blog post to clarify the negative effects of above-the-fold lazy loading and how CMSs can use heuristics to avoid it.

Since these best practices are applicable to all web developers, it may also be worth flagging lazy loading antipatterns in tools like Lighthouse. Refer to the feature request on GitHub if you're interested to follow along with progress on that audit. Until then, one thing developers could do to find instances of LCP elements being lazy-loaded is to add more detailed logging to their field data.

new PerformanceObserver((list) => {
  const latestEntry = list.getEntries().at(-1);

  if (latestEntry?.element?.getAttribute('loading') == 'lazy') {
    console.warn('Warning: LCP element was lazy loaded', latestEntry);
  }
}).observe({type: 'largest-contentful-paint', buffered: true});

The JavaScript snippet above will evaluate the most recent LCP element and log a warning if it was lazy-loaded.

This also highlights a sharp edge of the lazy loading technique and the potential for API improvements at the platform level. For example, there's an open issue in Chromium to experiment with natively loading the first few images eagerly, similar to the fix, despite the loading attribute.

Wrapping it up #

If your site uses native image lazy loading, check how it's implemented and run A/B tests to better understand its performance costs. It may benefit from more eagerly loading images above the fold. If you have a WordPress site, there will hopefully be a patch landing in WordPress core soon. And if you're using another CMS, make sure they're aware of the potential performance issues described here.

Trying out relatively new web platform APIs can come with both risks and rewards—they're called cutting edge features for a reason. While we're starting to get a sense of the thorniness of native image lazy loading, we're also seeing the upsides of how to use it to achieve better performance.

Photo by Frankie Lopez on Unsplash