Austin Z. Henley

Associate Teaching Professor
Carnegie Mellon University


Home | Publications | Teaching | Blog

Dear researchers: Is AI all you've got?

2/22/2026

Doodle saying 'Hey researchers! Listen!'.

This is a draft of my upcoming article in the Dear Researchers column in the Journal of Systems and Software (JSS). I'm a co-editor of the column, and we invite industry practitioners to contribute articles addressed to researchers about practical challenges in technology transfer. Email me if you're interested!


What important problems have software engineering researchers abandoned so that they could focus on AI instead?

In the early 1900s, bacteriophage therapy was a popular topic in medical and biological research for its potential to target specific bacteria1. However, after the discovery of penicillin in 1928, the research community shifted its attention to antibiotics, mostly abandoning bacteriophage research by the 1950s2. Interest surged again in the 2000s as antibiotic resistance grew more common and the highly targeted nature of phage therapy became more appealing3.

Is a similar phenomenon happening in the software engineering community? Are researchers over-indexing on AI? At ICSE '25, one third of research-track papers and two thirds of industry-track papers involved AI. In contrast, a study of research trends from 1992 to 2016 in software engineering did not even include AI or ML in the top-10 most popular topics4. Furthermore, AI conferences have grown exponentially, with AAAI going from 9000 submissions in 2022 to 29,000 submissions for the 2026 conference.

What might be missed?

My concern is not that AI doesn't have considerable value (I have worked on AI in both academia and industry) or that it isn't a huge innovation for the field, but rather that other topics are important too. What innovations might we miss? What problems will continue to go unsolved? What daily pain points exist beyond AI?

As software engineers build highly distributed systems, architectural and implementation decisions play a critical role in system resilience. In 2025, we saw several significant outages from Google Cloud, AWS, Azure, Cloudflare, and Cloudflare again. It felt like the entire internet was down! These outages impacted Spotify, Discord, ChatGPT, Zoom, Venmo, Reddit, Amazon, LinkedIn, Shopify, Fortnite, Square, banks, airlines, and even brick-and-mortar retailers.

Given that approximately 20% of all websites use Cloudflare and that AWS, Azure, and GCP account for 62% of the global cloud infrastructure, this is of huge concern. The AWS outage was estimated to have cost businesses upwards of $581 million dollars in just 15 hours and one of the Cloudflare outages to have cost $300 million dollars in less than 4 hours. In fact, I argue that it is a global catastrophe waiting to happen and we have seen repeated warning signs, yet I do not see researchers flocking to solve the problem as if lives depend on it. Has AI dramatically increased the reliability of software?

From my experience in industry, most recently at a startup that went through an acquisition, the problems we faced were fundamentally human problems. For example, convincing business partners, customers, and lawyers that our software does what we say that it does and that it won't have major disruptions was a huge challenge that we spent considerable time on. AI allowed us to scale faster, but it did not help us convince others to trust our system.

Looking over the keynote talks from ICSE and FSE leading up to the release of ChatGPT in late 2023, we can remind ourselves of the topics that were once top of mind for researchers: software safety, reliability, runtime monitoring, testing, industry impact, software engineering education, research rigor and reproducibility, socio-technical coordination, ethics and privacy, environmental impacts, etc. Several of these talks are relevant to the global cloud outages we have been experiencing, such as Marsha Chechik's keynote at FSE 2022 on the safety and reliability of software. Unfortunately, it seems that her call to action was not enough.

Follow the incentives

But perhaps there are innovations out there that we aren't even thinking about. Clayton Christensen, in his seminal book, Innovator's Dilemma5, argues that as fields mature, they often optimize and sustain existing innovations while also overlooking novel, disruptive innovations. He goes on to state that disruption often begins in places that others dismiss as too small, too immature, or too orthogonal, which are the exact areas that risk being neglected when AI becomes the default answer to every research question.

A straightforward explanation for why researchers would jump on the AI bandwagon lies in incentives. There has been an incredibly strong force pulling everyone into AI, including funding agencies asking for AI-related proposals, conferences adding multiple topics in AI to their calls for papers, and universities posting numerous faculty openings for AI researchers. It makes it incredibly difficult for a researcher to avoid. In fact, the papers I co-author involving AI are being cited at a far, far higher rate than any of my other work. As my co-editor, Olaf Zimmermann, observes, academic behavior is often shaped less by intrinsic motivation than by the incentive structures and metrics that govern how academics are evaluated6. In other words, "incentives, incentives, and incentives".

I'm certainly not the first to voice concerns about the fixation on AI, in fact, several others have argued about the potential downsides of AI. Johnson and Menzies stated that "AI overhype" is dangerous and that it is "the ethical duty of software professionals to rally against such remarks"7. Early studies of AI use in the classroom have shown evidence that it may disrupt learning8, 9, 10. This is further complicated by the disruption in the job market for software engineers, especially those that are early in their careers, caused by AI.

Right now is the opportunity

Given that virtually every researcher is focused on AI, now is the opportunity to make progress on any other topic. Even Yann LeCun, the 2018 Turing Award winner for his foundational work on deep neural networks, said that "an LLM is basically an off-ramp, a distraction, a dead end", in regards to his belief that LLMs will not continue to scale, and "don't work on LLMs.". He also warned, "right now, they are sucking the air out of the room anywhere they go, and so there's basically no resources for anything else. And so for the next revolution, we need to take a step back and figure out what's missing from the current approaches".

Perhaps we can use machine learning as inspiration to solve this concern of over-indexing on AI. Machine learning algorithms have mechanisms for escaping local maxima. Simulated annealing and stochastic gradient descent introduce noise, evolutionary algorithms create variation with random starting points and mutations, and optimization methods use random perturbations and restarts to explore new areas of a search space.

Can researchers apply these concepts to how they decide their next research topics? I worry that researchers are stuck in a local maximum. Worse yet, they are intentionally reducing the "randomness" in their exploration of research topics with the goal of joining (or staying on) the AI bandwagon.

While everyone is researching AI, the software systems that quietly run our society continue to fail in ways that are neither rare nor surprising. If the global cloud can go down repeatedly, affecting banks, airlines, retailers, and governments, then resilience and trust are not solved problems. The unfortunate part is not that these problems are hard, but rather that they increasingly feel unfashionable.

In the 2021 book Think Again11, organizational psychologist Adam Grant claimed, "Thinking like a scientist involves more than just reacting with an open mind. It means being actively open-minded. It requires searching for reasons why we might be wrong, not for reasons why we must be right, and revising our views based on what we learn." Whether it is through this scientific approach or through random perturbations, we can't get stuck in local maxima.

So software and systems researchers, is AI all you've got?


Special thanks to Olaf Zimmermann for providing multiple rounds of feedback to improve this article, and for being the co-editor of Dear Researchers with me.

If you have opinions that you want to share with the research community, consider writing an article for Dear Researchers. Reach out to me.

References

  1. Salmond, G. P., & Fineran, P. C. (2015). A century of the phage: past, present and future. Nature Reviews Microbiology, 13(12), 777-786.
  2. Wittebole, X., De Roock, S., & Opal, S. M. (2014). A historical overview of bacteriophage therapy as an alternative to antibiotics for the treatment of bacterial pathogens. Virulence, 5(1), 226-235. https://doi.org/10.4161/viru.25991
  3. Gordillo Altamirano, F. L., & Barr, J. J. (2019). Phage therapy in the postantibiotic era. Clinical microbiology reviews, 32(2), 10-1128.
  4. Mathew, G., Agrawal, A., & Menzies, T. (2018). Finding trends in software research. IEEE Transactions on Software Engineering, 49(4), 1397-1410.
  5. Christensen, C. M. (2015). The innovator's dilemma: when new technologies cause great firms to fail. Harvard Business Review Press.
  6. Zimmermann, O. (2025). Overcoming the research-practice gap: Root cause analysis and topics of practical relevance in software architecture and distributed systems. Journal of Systems and Software, 230.
  7. Johnson, B., & Menzies, T. (2024). Ai over-hype: A dangerous threat (and how to fix it). IEEE Software, 41(6), 131-138.
  8. Kazemitabaar, M., Williams, J., Drosos, I., Grossman, T., Henley, A. Z., Negreanu, C., & Sarkar, A. (2024, October). Improving steering and verification in AI-assisted data analysis with interactive task decomposition. In Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology (pp. 1-19).
  9. Bastani, H., Bastani, O., Sungu, A., Ge, H., Kabakcı, Ö., & Mariman, R. (2024). Generative AI can harm learning. The Wharton School Research Paper.
  10. Prather, J., Reeves, B. N., Leinonen, J., MacNeil, S., Randrianasolo, A. S., Becker, B. A., ... & Briggs, B. (2024, August). The widening gap: The benefits and harms of generative ai for novice programmers. In Proceedings of the 2024 ACM Conference on International Computing Education Research-Volume 1 (pp. 469-486).
  11. Grant, A. (2023). Think again: The power of knowing what you don't know. Penguin.