A claim that AI models are far easier to understand than researchers assumed has drawn wide attention in the AI safety field, though South African academics caution that the findings are unverified and not yet peer reviewed.
Nick Cammarata, a researcher at AI start-up Goodfire who previously worked at OpenAI, makes the case in Neural networks are already clean, billed as “an informal research essay and working hypothesis”.
“The canonical evidence of polysemanticity and superposition that convinced much of the field, including me, that networks pack concepts arbitrarily was misread,” he writes. Those terms describe the idea that individual neurons juggle several unrelated concepts at once.
He told MIT Technology Review: “We were looking at it wrong.”
Steven Boykey Sidley, visiting professor at Rhodes University, said AI’s black box nature, which even its builders cannot untangle, underpins much of the anxiety about it.
The Cammarata paper suggests that it is not that complicated inside the AI neural network – we have just been looking at it through the wrong lens.
Steven Boykey Sidley
“This has been unnerving, because if you can’t understand how AI thinks, then how can you prevent it from doing bad stuff, like urging someone to commit suicide? That is at the heart of what is called AI alignment,” he told TechCentral.
“The Cammarata paper suggests that it is not that complicated inside the AI neural network – we have just been looking at it through the wrong lens. His partial and (as yet) unverified results are tantalising, because they hold out the promise that we can make AI our subservient tool, rather than the uncontrollable and unruly child that it now is.”
Cats, cars and a V
The essay revisits a famous case in interpretability, the field that tries to work out what happens inside AI models. Neuron 4e:55 – a unit that activates when it detects a particular feature – in Inception v1, a 2014 Google image-recognition model, responds to both cats and cars. In Zoom in, a 2020 paper Cammarata co-wrote in work led by Chris Olah, now head of interpretability research at Anthropic, which he co-founded, it was held up as a textbook case of a neuron mixing unrelated concepts.
Cammarata now argues that what links them is a shared V shape: the pointed face of a cat, the front of a car seen at an angle. The neuron also fires for a pool table’s corner and a bullet train’s nose, while round shapes push it the other way. A 30 October revision to the essay makes clear that the neuron still detects cats and cars, with the V shape as the common feature.
“It probably took me five seconds,” he told MIT Technology Review, having studied the model “for probably thousands of hours”.
If most neurons have one main job, he argues, researchers can inspect models without first training a second model to untangle them.
Preliminary audits of Alibaba’s Qwen3-32B model suggest that roughly 90% of “neuron poles” – the positive and negative ends of a neuron’s activity – have one main job, about 7% have two related jobs and 2-4% are not yet understood, he says, though the figures depend on thresholds. A project he says he started at OpenAI in 2023 found only about 1.7% of neurons interpretable “by our most generous measure”.
Not yet peer reviewed
Prof Benjamin Rosman, founding director of the Wits Mind Institute, named among Time’s 100 most influential people in AI in 2025, called it “an interesting and potentially very encouraging development for AI safety”.
“However, the research has not yet been peer reviewed, and it is far too early to draw strong conclusions about what this means for AI safety,” he said.
Others are more sceptical. “This is interesting work, but I don’t think it’s a significant update to the big picture of interpretability,” Google DeepMind’s Neel Nanda wrote on X. Jacob Hilton of the Alignment Research Center added: “For a full understanding, we need to go beyond neurons, and explain how they connect together to form circuits.”
Rosman added: “Even if we become much better at understanding how these systems work, that does not necessarily mean we know how to make them behave safely.” – (c) 2026 NewsCentral Media
