Goodfire researcher says LLMs may be easier to understand

Nick Cammarata, a researcher at the startup Goodfire and a former OpenAI employee, argues in a 26,000-word post that large language models (LLMs) may be easier to understand—and therefore to monitor and control—than previously believed. The hypothesis has prompted speculation and celebration among AI researchers, with some describing the post as a possible scientific milestone.

Chris Potts of Stanford University, however, calls the material little more than a hunch and says it does not make the problems of understanding LLMs any easier. The prevailing view in the field is that neural networks have structures that are difficult to interpret, partly because a single neuron can represent different concepts, a phenomenon known as superposition. Cammarata helped develop mechanistic interpretability techniques but says he was previously pessimistic about the possibility of understanding large models in detail.