Austin Z. Henley

I work on software.

Home | Publications | Blog

The pain points of building a copilot


See the discussion of this post on Hacker News.

An illustration of engineers building a robot.

This post is an informal summary of our paper, "Building Your Own Product Copilot: Challenges, Opportunities, and Needs". See the preprint for more details. Thanks to my collaborators at Microsoft: Chris Parnin, Gustavo Soares, Rahul Pandita, Jessica Rich, and Sumit Gulwani.

Companies everywhere are launching copilots—AI assistants that leverage LLMs to assist with tasks. For example, Office Copilot helps you with Word and Excel, GitHub Copilot helps you write code, Intuit Assist helps with filing taxes in TurboTax, Adobe Firefly helps you generate and edit images, and Dropbox AI answers questions about your documents.

A screenshot of two example copilots. Left is Excel and right is Windows. Given the sudden surge in generative AI being integrated into products, we wanted to know what is the process that software engineers follow to build these products, what are the pain points, and what are the opportunities for tools.

Towards this, we conducted semi-structured interviews with 26 developers from a variety of companies that are working on copilots. We qualitatively analyzed their responses to identify themes. Then we conducted two focus group sessions with tool builders that involved reviewing our interview findings and brainstorming possible solutions.

Process of building a copilot

Given the nondeterministic nature of LLMs, it changes the typical software development process. We modeled the high-level process that the developers we interviewed are following: exploration, implementation, evaluation, and productization. It is not linear though, rather it is messy and iterative.

A chart showing the high-level process of building a copilot, with example activities for each step.

For example, there are additional risks that product teams are taking on. They must identify business scenarios that they think are feasible to solve using LLMs, tinker with various technologies to see if it works (e.g., LLMs, embeddings, vector databases), evaluate if it is good enough, and then they have to ship it to actual users.

Each and every one of these tasks has pain points for developers.

Pain points

We categorized the pain points into 6 fundamental themes that developers face while building a copilot:

Opportunities for tools

In our focus group sessions with professional tool builders, we identified several opportunities for tools, processes, and techniques to help developers build copilots.

It is still the wild, wild west of product copilots. It will be interesting to see how software engineering will evolve, either through new processes or tools, over the next several years.

An illustration showing hundreds of arguing about how to build a robot.

Read more in the paper: Building Your Own Product Copilot: Challenges, Opportunities, and Needs.