The future of programming: Research at CHI 2023

See the discussion of this post on Hacker News.

The esteemed CHI conference is happening this week, and I'm jealous that I can't be there.

Instead, I'm going through the proceedings and reading all of the papers related to programming, of which many involve AI.

Here are some papers that stood out to me:

"What It Wants Me To Say": Bridging the Abstraction Gap Between End-User Programmers and Code-Generating Large Language Models

preview video, talk video, paper

My summary: To help end users write LLM prompts, the researchers designed a system that maps a user's natural language input to a series of system actions. This communicates to the user precisely how the system is interpreting their prompt and gives a list of fine-grained actions that can be edited.

Why Johnny Can't Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts

preview video, talk video, paper

My summary: Prompt engineering is quite the craze right now, but can non-AI-experts write effective prompts? The researchers investigated the challenges that people face while writing prompts and designed a tool to help these non-experts do so.

VizProg: Identifying Misunderstandings by Visualizing Students' Coding Progress

talk video, paper

My summary: It is challenging for instructors to understand how students are doing during in-class activities. The researchers designed a tool that visualizes the students' different approaches in real time. It shows an edit-distance view of the entire class while also allowing an instructor to check specific submissions.

On the Design of AI-powered Code Assistants for Notebooks

preview video, talk video, paper

My summary: This is an exploration of the design space for AI code assistants in computational notebooks. They provide a useful framework for the UX of AI, that includes how users iterate on their input, how they refine the output, and much more.

Colaroid: A Literate Programming Approach for Authoring Explorable Multi-Stage Tutorials

talk video, paper

My summary: Even a short programming tutorial can take many hours to do well. The researchers designed a tool for creating interactive notebooks-based tutorials. It allows the author to grab code chunks from their codebase's history, then remix them for better explanation.

Studying the Effect of AI Code Generators on Supporting Learners in Introductory Programming

preview video, paper

My summary: The world wants to know how AI is going to change education. Will over-reliance get in the way of learning? The researchers conducted a 3-week study and found that students using an AI tool for their tasks did not perform worse on manual coding tasks or on tests. Perhaps AI is the new calculator after all.

Log-it: Supporting Programming with Interactive, Contextual, Structured, and Visual Logs

preview video, talk video, paper

My summary: Print statements are still a popular means of debugging, but it isn't always the most effective technique. The researchers designed a snazzy tool to structure and visualize output on the fly.

Structured Editing for All: Deriving Usable Structured Editors From Grammar

preview video, talk video, paper

My summary: Structured editors is one of those ideas that everyone seems to love and yet they haven't taken off. The researchers focused on making a structured editor that is available to all programming languages while still being highly usable. It provides a consistent UI that can work on virtually any language given the formal grammar.

I'll definitely be at CHI next year in Hawaii...