🔍 Read the full analysis: What This AI Experiment Reveals About Neural Network Weights on ThorstenMeyerAI.com
Get business pricing on monitors, keyboards and dev gear
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
TL;DR
MIT CSAIL researchers report that OpenAI’s GPT-6 Astra interpreted and edited neural-network weights directly, without a training loop or gradient descent. Their public logs include successful small-network repairs and networks written from scratch, but the work is a blog-post report with limited experiments, and some tests failed.
Researchers at MIT CSAIL report that OpenAI’s GPT-6 Astra could inspect, modify and create small neural networks by working directly with their weights, rather than using conventional training. The experiments, described in a November 2 blog post, include a car-control model repaired from zero successes to 1,024 successful episodes out of 1,024, alongside failures that show the approach’s current limits.
The researchers call the method weight-space programming: a language model receives the numerical weights that define a network and reasons about how to alter them. In some tests, Astra received the weights as text and responded in a single turn. Other experiments let it write Python code that ran inside a restricted research sandbox. The team says Astra had no shell, web search or file-editing tools, and that it reviewed 1,125 turns in the released logs for tool use.
The team reports successes across three kinds of task. Astra identified the games represented by four small game-playing networks and inferred the shape represented by three small image generators. In another test, it identified copying, or induction, heads in three tiny language models from their weights without running the models. These are results on the particular networks studied; they do not establish that weight-only interpretation works reliably for larger or unrelated systems.
For repair, the authors say Astra changed 3.5% of a car-control network’s weights, after which the model completed 1,024 of 1,024 test episodes; before the change, it completed none of 256 episodes. In a separate test, an edit to 0.016% of Qwen3.5-0.8B’s weights accompanied an increase in its MATH-500 score from 7.9% to 48.1%. The source reports that a Hugging Face engineer rebuilt the edited Qwen model using the public logs and obtained 51.0% on MATH-500.
The researchers also report creating small game-playing networks from all-zero weights, and a 1.5-million-weight network for 9×9 Go. After 50 test runs, the Go network beat a same-size comparison network trained on about 20.5 million games. The source says some newly written game networks outperformed comparison networks trained for 100 million steps. Those comparisons apply to the specified tests, not to game-playing systems generally.
Infographic 01 · Eight experiments
Read, fix, write: what GPT-6 Astra did with raw weights
Every experiment in the weight-space programming post, with the size of the network, the result and how it went.
Read
- 1Which game was this network trained on?4 reinforcement-learning policies · 17,253 weights each · weights as text, no toolsRight game for 4 of 4, without a list of options✓ Worked
- 2What does this image model draw?3 tiny diffusion models · 23,824 weights · no forward pass allowedRight shape for 3 of 3, but drawn as outlines; the models draw filled shapes✓ Worked
- 3Which language did this model learn?3 character-level transformers · 7.16M weights · middle blocks only, never runRight language 3 of 3; every copying head found, none invented✓ Worked
- 7Which of two models is sycophantic?3 pairs of Gemma 3 270M fine-tunesWeights alone: could not tell in 4 attempts. With attention patterns: 3 of 3! Partly
Fix
- 4Repair a car that never backs upKinetix control network · 171,664 weights · one reply, no tools0 of 256 episodes before, 1,024 of 1,024 after; 3.5% of weights changed✓ Worked
- 8Make Qwen3.5-0.8B better at maths0.8 billion weights · Python over the weights · 10 / 25 / 50 test passesMATH-500 7.9% to 48.1% with 50 passes (12.3% and 14.1% with 10 and 25)! Partly
- 8bSame task, a model that does not loopGemma 3 270M · 5 to 50 test passesMATH-500 6.8% to at most 7.7%: “not established”✕ Failed
Write from zero
- 5Write arcade-game networks3 MinAtar games · 20,486 weights, all starting at 0 · one reply, no toolsBeat the 100M-step comparison networks in 3 of 3, the 500M-step ones only in Breakout! Partly
- 6Write a 9×9 Go network1,480,274 weights, all starting at 0 · no training, no search0 passes: 54% vs small trained nets. 50 passes: 70–79% vs same-size nets trained on ~20.5M games✓ Worked
✓ Worked · ! Partly or with conditions · ✕ Failed. One run per setting, no error bars. A blog post with complete logs, not a peer-reviewed paper.
Sources: Takehi, Teoh, Zhang, Khattab (MIT CSAIL), blog post, 2 Nov 2026, lm_understand-logs (CC BY 4.0). Model: OpenAI GPT-6 Astra via the Codex CLI, built-in tools switched off.
Powered by Thorsten Meyer AI
A New Route to Targeted Model Repairs
If the results hold up in broader testing, direct editing could offer researchers and developers another way to inspect small models and make targeted changes, particularly when retraining is costly or when a specific failure is understood. The Qwen result is notable because the reported edit touched a very small share of its weights while its benchmark score rose substantially. However, the experiment does not establish why the edit produced the change, whether it will persist across other evaluations, or whether the method can be used safely on deployed systems.
The work also highlights a practical concern: models can be changed through their weights, so organizations may need to track where model files came from and verify that they have not been altered. That is a possible implication, not a security finding demonstrated by the experiments. For most users, the report does not change how AI products work today; the tests centered on small models and controlled research settings.
Popular right now
De'Longhi Rivelia Fully Automatic Espresso Machine with Milk Frother, White
Vitamix 5200 Professional-Grade Blender, 64 oz. Container, Black
Dyson Big Ball Multi Floor Canister Vacuum, Yellow
As an affiliate, we earn on qualifying purchases.
From Optimizer Training to Weight Editing
Neural networks are built from numerical weights that shape how information moves between artificial neurons. Standard training adjusts those numbers repeatedly using an optimizer and data. The MIT experiments tested a different idea: whether a general-purpose language model could infer some structure from the numbers and change them directly, without gradient descent.
The source connects the idea to Andrej Karpathy’s 2017 description of neural networks as “Software 2.0,” in which optimization writes programs into weights rather than people writing conventional code. The MIT team’s November 2 post extends that framing to language-model-driven inspection and edits. The researchers are Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab; Takehi and Teoh are described as equal contributors.
The account is based on a research-team blog post with public logs, not a peer-reviewed paper, and the source says it is not posted on arXiv. It reports one run per setting, small networks, a single model family per task and one research group. Those constraints make the results useful as an early demonstration, but leave broad claims about generality untested.
““not through an optimizer, and in particular not with gradient descent””
— MIT CSAIL researchers, in their November 2 blog post
How Far the Tests Can Generalize
The report leaves open whether the method works across larger models, different architectures and tasks beyond the examples tested. The experiments used one run per setting, and the source notes that each task involved one model family. The results also do not show whether edits remain reliable across repeated runs or when tested on broader sets of prompts and conditions.
There are direct counterexamples in the report. Astra could not distinguish an honest model from a sycophantic one using weights alone, and edits did not improve maths performance on Gemma 3 270M. The authors say editing could not add a capability the model did not already have. The successful score changes also do not, by themselves, explain what changed internally or whether the gains involve trade-offs on other tasks.
The work has not been described in a peer-reviewed paper, according to the source article. Public logs and an external reconstruction of the Qwen edit provide ways to inspect some results, but they do not replace independent evaluation across more models and settings.
Independent Tests Will Set the Scope
The next evidence to watch for is replication by other groups, testing on additional model sizes and architectures, and evaluations that check whether gains survive broader benchmarks without harming other capabilities. The source provides no scheduled follow-up paper or experiment, so the timing of that work is unknown.
Researchers and developers can examine the released logs and the reported Qwen reconstruction, while treating the findings as an early demonstration rather than a general-purpose repair method. Until broader evidence is available, it remains unclear how often a language model can diagnose weights correctly, make a useful edit and preserve performance elsewhere.
Key Questions
What is weight-space programming?
It is the approach tested by the MIT team in which a language model reads or edits another neural network’s numerical weights, rather than relying on a conventional training loop and gradient descent.
What did Astra successfully repair?
The researchers report that Astra edited a car-control network that had completed zero of 256 episodes; after the edit, it completed 1,024 of 1,024 episodes. They also report a large MATH-500 score increase after a small edit to Qwen3.5-0.8B.
Did the method work in every test?
No. Astra could not distinguish an honest model from a sycophantic one from weights alone, and its edits did not improve maths performance on Gemma 3 270M. The authors say editing could not add an ability the model lacked.
Has the research been peer-reviewed?
The source describes it as a blog post with public logs, not a peer-reviewed paper, and says it had not been posted on arXiv. The experiments were limited to one run per setting and small networks.
Does this change how AI products work now?
The report does not indicate a change to consumer AI products. It describes controlled experiments on small models; whether direct weight editing can be applied reliably to larger or deployed systems remains unclear.
Source: ThorstenMeyerAI.com
