Google's Devices and Services Marketing (DSM) organization oversees the go-to-market strategies for a wide range of products: smart phones, watches, earbuds, tablets, smart home devices, and relevant subscriptions, all of which are available both through the Google Store and global third party vendors. People learn about these products online and in physical retail stores.
One constant challenge for this team is educating their consumers and retailers about the use cases and benefits of the Google hardware ecosystem and advanced AI experiences. To better help consumers understand the products and functionality, the DSM team created a 3D virtual space with advanced web technology to address many of these challenges. In this case study you can learn how the team created a more immersive experience for customers, launching these new experiences four times faster and at half the cost of traditional digital assets.
The challenge: product education
Educating sales associates and customers who are new to Google hardware products on the benefits of features such as interoperability and AI is costly and difficult. Traditional product education strategies rely on digital content that is produced using physical products, then hosted on digital learning platforms. These learning platforms provide product technical specifications, images and videos, but not access to physical or connected products.
Learning content such as video is expensive to produce, complex to localize for global markets, and near impossible to reuse between products. Creating the initial assets requires budgets for casting, wardrobe, location scouting, location fees, studio fees, film crews, and post production work. Production costs and deliverables also need to account for localization; modifying the assets, locations, products, copy, and talent is especially challenging for traditional marketing to manage at scale. And when you consider that most of the products supported have feature drops every couple of months, these assets are already outdated by the time they're finished.
Exploring better ways to share product information
Trying to find a better way to share product information, the DSM team experimented with VR/AR experiences in an app. The results were promising with both strengthened engagement and increased basket sizes at points of sale. However, app downloads were a significant barrier to entry for both sales associates and customers, and limiting the experience to an app meant it wasn't embeddable in other first or third party properties like store.google.com.
Lightweight solutions with <model-viewer>
Having seen the success of 3D product models for product education, the team
decided to bring this approach to the web. One way to accomplish this is using
<model-viewer> to create 3D experiences
with individual products.
<model-viewer> is a web component,
which lets you declaratively add a 3D model to a web page, while hosting the
model on your own site.
You can see this in action on the
Pixel Fold
page on the Google Store, where <model-viewer> allows users to see the Pixel
Fold from any angle with various folded positions. It was easy to integrate the
3D model into the rest of the HTML UX, with buttons to tell a story through
camera angles and interactive color variants. With <model-viewer>, you can
give your users any kind of experiences you can dream up.
Creating the 3D models
The first step to developing a 3D virtual experience is to create the 3D product models. The DSM team created their 3D models in CAD. By working closely with the designers producing product CAD, the DSM team was able to export low poly renders that could be optimized for the web. Some of the best practices they used to achieve this were in the following areas.
- Geometry:
- Focus on making the geometry (the shape and scale) accurate from every angle.
- Avoid using normal maps for beveling edges.
- Create decals as separate geometry.
- Colors and materials:
- Goal: consistent visual representation of physical properties.
- Consider real-time lighting dynamics.
- Use separate texture sets and materials for every mesh type (Opaque, Transparent, Decal).
- Iterate on designs with the original CAD designers if further adjustments are needed.
- File size:
- Export as a low poly model in GLB format so that the model can be used
by
<model-viewer>. - Compress the geometric meshes manually by a 3D designer, with a vendor, or through compression software like DRACO (OS).
- Export as a low poly model in GLB format so that the model can be used
by
Because these 3D models would be used often on mobile phones, they were optimized by setting a maximum file size with textures of 2 MB, to support older generation devices and weak internet connections.
<model-viewer>
The DSM team uses Google's <model-viewer> open source web component to embed
the newly created 3D models into their webpages. In order for the models created
in step one to be compatible with <model-viewer>, the assets need to be in a
gITF or GLB format (extension .glb). Both formats are compact, compressible, and
load rapidly into the GPU. The <model-viewer> component is supported by all
major evergreen browsers.
During this step, the biggest challenge the DSM team encountered was that the Google hardware color palette didn't render accurately. The team eventually discovered that ACES tone mapping (a film industry standard) was responsible for the lost colors. To solve this, they developed a new Khronos PBR Neutral Tone Mapper specifically to address these deficiencies and present colors precisely to the screen, while avoiding the blown-out highlights and hue shifts associated with linear tone mapping.
<model-viewer src="Pixel8Pro_Bay_enUS.glb" ar ar-modes="scene-viewer webxr quick-look"
camera-controls poster="poster.webp"
shadow-intensity="1" tone-mapping="commerce"></model-viewer>
To learn more about <model-viewer>, visit
modelviewer.dev. To test your 3D models and download
a whole starter website, try our editor.
Heavyweight solutions with three.js
<model-viewer> is a great, performant way to display and interact with one 3D
model. However, sometimes the DSM team needed a more immersive and
interconnected web experience than is possible with <model-viewer>. One
example was the launch of the Nest Mini + C. <model-viewer> could let
potential customers experience a product in 3D on the web, but it couldn't show
off its usefulness when combined with other Google Hardware products and AI
features like the assistant.
For this task, the team turned to the library underpinning <model-viewer>,
three.js. Three.js is an open source JavaScript game engine, and the team was
able to create a framework of reusable assets for a virtual home environment
containing Nest indoor cameras, lights, and speakers. This gave the team a
foundation to provide real time feedback on how the devices interacted with each
other.
Dialogflow
The last piece of creating the interconnected experience is to add Google Assistant. This meant users could try authentic commands and routines with the interconnected virtual experience. However, inserting the Google Assistant from the user's existing account would open up a number of problematic privacy concerns. To create a privacy-forward solution the DSM worked with the Google Cloud Dialogflow service to mimic the Google Assistant in this space. The following high level diagram shows how the web app used Dialogflow's API (adapted from Dialogflow Basics). For each conversational turn, the web app used Dialogflow's intent classification and the API returned predetermined end user expressions that matched the intent.
To learn more about Dialogflow, visit the Google Cloud documentation.
End result: an embeddable iFrame
The end result is a library of assets that can be embedded into a web page with
<model-viewer> or used in a full virtual environment to help customers learn
more about individual products and how the products interconnect. The experience
is both authentic, scalable, and cost effective. This first virtual experience
launched in May 2021, and while it's no longer up on Google Store's website you
can still
try it out.
The results
Since the launch of Nest Mini +C, the DSM has been able to reuse and expand the framework for the past two years of Pixel portfolio launches with increasing success. Through operationalizing the iteration of these 3D assets and experiences, the team has so far been able to quadruple the number of interactive product education experiences and triple the number of products benefiting from this web technology.
The end result is a branded, authentic product education for a growing permutation of use cases at scale that is more sustainable, cohesive, and interactive than previous strategies. And looking forward, the DSM team now has a robust portfolio of components of an immersive experience that they can quickly adapt to dynamic business targets. These improvements allow the DSM team to launch new product education content four times faster and at less than half the cost than their earlier, more traditional processes.