Does correctness still matter?
FSE 2026 Keynote vs 2025 ACM SIGSOFT Outstanding Research Award
The answer is not clear, as large language models (LLMs) become more and more popular. Regardless of “hallucinations“, or mistakes, being a feature of LLMs, more and more discussions, deployments, and investments, continuing the excitements of ChatGPT.
In a keynote in the ACM International Conference on the Foundations of Software Engineering (FSE) 2026, CMU Prof. Mary Shaw argued that: Software engineering has a complicated relationship with “correctness”.
Correctness, confidence, and context: Framing software assurance in the AI age
She presented AI Challenges and Inherent Limitations: First, a great deal of tacit knowledge is not available in a form that makes it accessible for LLM training sets. Second attempting to go directly from natural language problem descriptions to running code short-cuts the world-machine mapping. Third, the statistical basis of machine learning is unsound; it does not permit rigorous reasoning, and SE has not yet come to understand how to manage this difference.
MIT Prof. Martin Rinard received the 2025 ACM SIGSOFT Outstanding Research Award, ”for fundamental contributions in pioneering the new fields of program repair and approximate computing.”
Research in Program Repair and Approximate Computing: A Retrospective
From Abstract: The prevailing value system in the field at the time focused on program correctness as a fundamental goal. This research, in contrast, was driven by a new perspective that emphasized acceptable (but not necessarily fully correct) survival through errors and the automatic identification and exploitation of performance versus accuracy tradeoff spaces implicitly present in computations coded to operate at only a single point in this space.
From Introduction: It is difficult to convey just how much skepticism this direction initially inspired. The programming language community in particular was deeply committed to correctness as a fundamental goal and reacted very negatively to the concept of executing through errors, especially with manufactured values.
Also from Introduction: The computer systems and software engineering communities were also skeptical but also more open to the idea — indeed, I have consistently found that one of the strengths of the software engineering community is the value it places on new ideas and concepts and its willingness to consider ideas that go against the current value system. Because of this openness, many of the results that I am most proud of have been published in the software engineering literature.
From both Abstract and Introduction, Prof Martin Rinard quoted from an anonymous reviewer: The basic idea—to assist incorrect programs in their efforts to emit incorrect output—is an abomination and if adopted would likely usher in a new dark age.
As LLMs become more and more popular, does correctness still matter, for communities like programming language, formal verification, and theory of computation?
As LLMs become more and more popular, does correctness still matter, for fields like computer science, engineering, and natural science?