Like many of you, we've been having lots of conversations about artificial intelligence and the future of the web. There's a lot of noise, and it's hard to know what exactly we, as web developers, need to know. The web is so much bigger than our team, so we wanted to gain some clarity around how you're thinking about AI, what you hope to learn, and what you want to do with these emerging technologies. That way, we can best provide you with content to break through that noise.
Over the last few months, we've spoken with web practitioners to understand the landscape and how you think about AI. Of course, we can't talk to everyone about everything. We had just a small set of conversations with web developers, including web Google Developer Experts, focused on how developers are using AI to deliver user-facing features and to increase productivity in their daily workflow.
We believe that what we learned may be broadly applicable to other folks on the web. And, we thought the community may be interested in hearing what we've learned.
We took this feedback, along with other research of the web developer landscape, to create our new AI collection. In this collection, you'll find overviews geared towards web developers, codelabs and demos, and other resources for thinking about AI tools and models on the web.
And this is just the beginning. You'll see much more from us in the coming months.
Improved productivity with generative AI
We noticed that web developers want to take advantage of generative AI to increase their productivity and interact with chat bots to learn new technologies or seek answers to their web development questions.
The developers we talked to are either already using AI in their daily workflow, for business or personal projects, or know someone who is.
Code generation
We learned from you that code generation tools, such as Gemini and Copilot, are great for standard unit tests, basic autocomplete (where you know what to write, but just need to write it), and simpler functions that don't require expansive knowledge of the codebase. These tools tend to be less useful when it comes to writing more complex, algorithmic code and functions that require broad context of a specific project.
More senior developers mentioned a concern around the long-term quality of their codebases, including issues around code duplication and long-term maintainability. Some were concerned that less experienced team members may not detect errors or know how to accurately validate the code produced by generative AI tools.
Developers have also shared that use cases which require specific domain knowledge, such as writing accessible components, aren't yet accommodated correctly by code generation tools they've tried.
Learning with LLMs
We've seen many developers using ChatGPT and Gemini to learn software development concepts, like asking a large language model (LLM) to explain how a sorting algorithm works, ramping up on different programming languages, or closing knowledge gaps.
You think LLMs provide a great experience due to the quick interactivity of questions and responses, and because AI won't criticize the questions being asked, and just provide the answer they need.
Once again, there is a concern related to more junior developers, where a minimal level of knowledge is necessary to catch the instances where the model is hallucinating and generating incorrect responses.
IP protection as a business concern
Many of the developers we spoke with said their companies don't yet have policies around developers using generative AI to increase their productivity. Usage of generative AI tools is often driven by the developers experimenting with it.
"My company generally misunderstands what AI means, so they're not creating the right policies."
However, businesses that do have a policy tend to discourage usage, due to concerns around leaking the company's intellectual properties (IP) to third parties. There are cases where such policies have been changed, after engaging directly with the companies behind these tools to understand how the data is used and potential risks.
With enterprise accounts and partnerships which are dedicated to ensure data protection, businesses are more likely to encourage developer usage.
Generative AI for user facing product features
On the product side, it wasn't a surprise to us that, when we prompted our conversation with the term "AI / ML," the response was often focused on generative AI. Developers are curious on how generative AI can be used to improve the experience for their users, but are unsure about what those experiences look like and what are the tools available to deliver them in production.
For those developers that have built or are building generative AI features into their products, using generative AI to answer user questions, with chatbots or one-off interfaces, is the most common use-case.
Output quality is the top concern we heard from you. In particular, developers hope to ensure the responses are accurate and aim to prevent the LLM from generating content unrelated to the intended goal. This is particularly true when the LLM's output is directly user-facing, such as a chatbot.
"Demos with AI are wild. Everytime I demo my project, the output is completely different."
You're putting a lot of effort into creating test suites to validate generative AI outputs for a wide variety of prompts, but there's no clear and established way to test or monitor the responses. Most of the evaluation work is manual. Many developers are new to handling non-deterministic outputs. As a community, we haven't yet built systems which work well with them.
The cost of running a generative AI model is also an important concern, and developers are carefully evaluating the cost against the benefits for their users.
Standard versus customized models
Most commonly, the developers we spoke to tended to rely on ready-made models and APIs. This optimizes the time-to-market and the use of engineering time and knowledge, which were limited.
"I want to stay in the web development space. I don't want to become an ML engineer."
While developers are aware of and see potential value in advanced techniques, such as Retrieve Augment-Generate (RAG) and fine-tuning, you'd rather focus on the web development aspect of their work. Ultimately, you prefer to use the default tools or rely on other teams to produce optimized models for their use cases.
Privacy and security concerns
Privacy and security were the top concerns, especially for verticals with strict data requirements, such as the medical industry. On-device AI may be key to address those use cases, but this area remains largely unexplored.
Exposing user data to more third-parties with cloud APIs is a concern, and a number of developers see the value of on-device machine learning or generative AI to mitigate potential privacy and security pitfalls.
AI for web developers
AI is everywhere and evolving at an incredible pace. How can we stay up to date, incorporate existing tools and models, or work with ML engineers to produce new models that best suit our needs?
Based on what we learned from you, we're working on guidance on AI for web developers. We aim to help you understand AI concepts on a high level, discover opportunities to use generative AI for productivity gains, and use AI to build delightful user experiences, using existing tools, models, and APIs. Continue to check back in as we publish more content in our AI collection.
While most web developers prefer to keep their focus on doing what they do best (that's web development!), we encourage those of you who want to dive deeper to build the tools, models, and APIs web developers need. We want to hear from you and learn how we can help you be successful.
AI is a fast-moving area. So, we'll continue engaging with the community as things change, running more conversations and surveys. If you'd like to discuss with us, schedule office hours with our team.