Lighthouse evolution: continuous integration, new performance score formula, and more

Get the latest Lighthouse updates and insights from Chrome Developer Summit 2019.

Elizabeth Sweeny

In the Speed Tooling Evolutions talk at Chrome Developer Summit (CDS), Paul Irish and I presented the newest products and features coming from Google that can help you build and maintain an exceptionally fast experience for all your users. At the center of that story are additions to the Lighthouse family of performance monitoring tools.

Lighthouse CI alpha release

The Lighthouse team has launched the alpha version of Lighthouse CI the new continuous integration product that enables you to run Lighthouse on every commit before pushing to production. Lighthouse CI runs Lighthouse multiple times, asserts static audit or metric thresholds, and then uploads Lighthouse reports to a server for visual diffing and basic category score history. Existing budgets.json configurations work seamlessly alongside the new expressive syntax for asserting any Lighthouse audit or category result.

Lighthouse CI supports Travis CI, Circle CI, and GitHub Actions out-of-the-box and any Ubuntu or container-based CI service with some configuration. You can install the Lighthouse CI server on-premise or use a docker image for instant setup. Free, public, temporary Lighthouse report storage is available as an alternative to get started right away.

Coming soon: Performance score updates

Changes are coming to the Lighthouse Performance score version 6! In version 5 (as of November 2019), Lighthouse has five metrics that are weighted and blended to form the 0-100 Performance score: First Contentful Paint, Speed Index, First Meaningful Paint, Time to Interactive, and First CPU Idle.

Comparison of Lighthouse performance score formulas in versions 5 and 6.

See Lighthouse performance scoring for detailed information.

In Lighthouse version 6, new metrics, Largest Contentful Paint (LCP) and Total Blocking Time (TBT), are replacing First CPU Idle (FCI) and First Meaningful Paint (FMP). The weights of each of the five metrics will be adjusted to better balance different phases of load and interactivity measures.

The Lighthouse team is still working to ensure that all scoring curves are fine-tuned, and the metrics are mature and thoroughly tested. They aim to ship the Lighthouse v6 Performance score in January 2020.

Note: Performance engineers sometimes find speed tools results difficult to reproduce due to two discrete challenges--variability and cross-environment inconsistency. Variability is the issue of seeing numbers change even when the testing environment remains the same. Cross-environment inconsistency is the issue of getting different results when running tests on the same page, but in differing environments (for example, DevTools and PageSpeed Insights). While the Lighthouse team is working on ways to mitigate variability, it's helpful to understand sources of variability and how you can deal with it. The Lighthouse team is also investigating calibration methods to reduce differences between environments, but it's fair to expect that different conditions and hardware lead to different measurements—at least for now.

Lighthouse Stack Packs

Lighthouse can automatically detect if sites use a framework or a content management system (CMS) and include stack-specific advice in the report. Stack Packs add customized recommendations, curated by community experts (like you!), on top of Lighthouse report core audits.

Lighthouse report recommendation for deferring offscreen images in React applications.

At the moment, there are Stack Packs for Angular, WordPress, Magento, React, and AMP. To create your own Stack Pack, visit the GitHub repo or contact the Lighthouse team.

Coming soon: Lighthouse plugins as Chrome Extensions

Lighthouse Plugins are another way you can take advantage of Lighthouse's extensibility. There are a lot of quality checks that Lighthouse core audits currently don't cover, either because they are only applicable to a subset of developers or because the team hasn't had the bandwidth to create the audits yet.

Lighthouse plugins allow community experts to implement a new set of checks that Lighthouse can run and add to the report as a new category. Right now, plugins only work in Lighthouse CLI, but the goal is to enable running them in the DevTools Audits panel too.

Chrome DevTools Audits panel with options for running Lighthouse plugins for Google Publisher Ads and User Experience. — Community Plugins in DevTools Audits panel (beta)

When users install Lighthouse plugin extensions from the Chrome Web Store, DevTools will identify installed plugins and offer them as an option in the Audits panel. The Lighthouse team will be building the support for the plugin approach in the coming months, so stay tuned. In the meantime, you can create a plugin today as a node module and make it accessible to all Lighthouse users via the CLI!

Learn more

For more details about Lighthouse and other performance tooling updates from CDS 2019, watch the Speed tooling evolutions talk:

Your feedback is invaluable in making Lighthouse better, so go on and try out Lighthouse CI, write a Stack Pack, or create a Lighthouse Plugin and let us know what you think.