🔍 Read the full analysis: We Tried Weight Space Programming—Here’s What Happened on ThorstenMeyerAI.com
Get business pricing on monitors, keyboards and dev gear
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
TL;DR
A November 6 report describes MIT CSAIL experiments in which GPT-6 Astra analyzed and edited neural-network weights without gradient descent, plus three further tests by the report’s author. The reported results range from no useful change to improvements on narrow tasks; the methods and scores have not been independently verified here.
A November 6 report says MIT CSAIL researchers used GPT-6 Astra to inspect, edit and generate neural-network weights without a training loop, a method the researchers call weight-space programming. The report also describes three separate trials by its author, with mixed results that suggest the approach may help with targeted changes but do not establish broad model improvements.
In the experiments described in the report, GPT-6 Astra received the numerical weights of other networks and was asked to infer their functions or make changes. According to the account, it identified the games behind four small game-playing networks from their weights and found copying heads in three small language models. The report says the researchers also asked it to create game-playing networks from all-zero weights; those networks beat comparison networks trained for 100 million steps in each of three games. These are results reported by the source, not independently confirmed in the material provided.
The report highlights an edit to the open model Qwen3.5-0.8B: it says the AI removed one direction of a looping behavior by changing 0.016% of the weights. On the MATH-500 evaluation, the reported score rose from 7.9% to 48.1%. The source says the approach did not help a model that lacked the looping behavior in the first place, and frames the technique as repairing an existing behavior rather than adding a missing capability.
In the author’s own trials, Claude’s edit reportedly increased the number of legal answers completed by a local model from 4 of 15 to 12 of 15, while its legal score remained zero. A second AI review identified faults missed by green tests. A separate one-matrix edit raised a score on a narrow JSON test from 49 to 70 out of 120. The source gives no independent replication or full methodological detail for these trials, so the results should be read as preliminary observations.
Infographic 01 · What weight space programming showed
Read, write, edit: what GPT-6 Astra did with raw weights
A blog post with complete public logs, not a peer-reviewed paper. Small networks, one research group, one run per setting.
MATH-500 score of the small open model Qwen3.5-0.8B after GPT-6 Astra removed one loop direction from 0.016% of its weights. Before: 7.9%. A Hugging Face re-run of the edited model: 51.0%.
Three things an AI did with raw weights
No training loop, no gradient descent
- Read: named the game behind 4 of 4 small game networks (17,253 weights each) and found every copying head in three small language models with their first and last layers removed.
- Wrote: arcade-game networks from all-zero weights that beat networks trained for 100 million steps in all three games. Breakout: 517.4 against 25.3 for 500 million steps.
- Fixed: a car network that never backed up: 0 successes in 256 episodes before, 1,024 of 1,024 after, with 3.5% of the weights changed.
Fixes what is broken, adds nothing new
Gemma 3 270M rarely loops (8.5% of answers hit the limit). No edit moved its MATH-500 score by more than about a point.
Some things stay invisible
Honest model or yes-man? From the weights alone: not found in 4 of 4 attempts. With a view of the running model: found in 3 of 3 pairs.
Sources: MIT CSAIL blog post, 2 Nov 2026 · logs on GitHub · Hugging Face re-run.
Powered by Thorsten Meyer AI
Small Edits, Narrow Gains
If the reported findings hold up, weight-space programming could offer developers a way to inspect or adjust an open model without repeating a costly training process. The reported change affected a small share of the weights, while some evaluations showed better performance on specific tasks. That makes the idea relevant to model debugging and auditing, especially where teams can examine model parameters directly.
The same capability could create safety concerns. The report warns that an edit could also remove behavior associated with safety training. Direct access to weights may make models more adaptable, but it also puts more importance on where model files come from and whether changes can be tracked. The source does not establish that these methods are ready for routine use or that they work reliably across model types.
Popular right now
Breville Bambino Espresso Machine, Brushed Stainless Steel
Breville BES870XL Barista Express Espresso Machine
Samsung 75-Inch The Frame Pro 4K Neo QLED Smart TV (QN75LS03FW)
As an affiliate, we earn on qualifying purchases.
From Training to Direct Edits
Neural-network weights are numerical settings that shape how a model processes inputs. Standard training adjusts those settings repeatedly, commonly using gradient descent, based on how far a model’s answers differ from a target. Weight-space programming instead asks a general-purpose AI to reason about the weight values themselves and propose changes, without gradient descent, according to the MIT post described in the source.
The report says the MIT work was posted on November 2, 2026, by Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab; Takehi and Teoh are listed as equal contributors. It also notes that earlier mechanistic-interpretability work had found some algorithms directly in model weights, though the report characterizes that work as requiring expert tools and time. The MIT effort’s stated aim is to have language models perform some of that reading and editing through deliberate reasoning.
“Our neural network weights will soon be natively optimized by LLMs that deliberately reason about their weights.”
— Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab, as quoted in the MIT post
Replication and Scope Remain Open
The source material does not provide enough detail to independently assess the experiments’ prompts, evaluation setup, or statistical variability. Although it says the MIT team published prompts, replies and evaluations on GitHub, the supplied account does not include the underlying materials or external replication. It is also unclear how the results compare across different model sizes, architectures and tasks.
The author’s three trials are reported as personal experiments, not controlled studies. In one, the legal score stayed at zero despite more completed answers; the JSON result concerns a narrow test. Those outcomes do not show general improvement in reasoning, legal accuracy or reliability. The report’s safety warning is a potential risk, and the source does not document a specific safety-training removal test.
Independent Tests Will Matter
The next useful step is independent replication of the MIT evaluations and the author’s edits, with clear documentation of model versions, prompts, baselines and scoring. Tests across more architectures could show whether weight-space programming is limited to particular behaviors or can support repeatable model maintenance.
For now, readers and developers should treat the reported gains as early, task-specific results. The source does not name a scheduled follow-up study or release date. It says the MIT team made experiment materials available on GitHub; further evaluation and scrutiny of edited model weights will determine whether the technique proves useful beyond these demonstrations.
Key Questions
What is weight-space programming?
It is a method in which a general-purpose AI analyzes or changes a neural network’s numerical weights directly, rather than relying on a conventional training loop for the change.
What did the MIT experiments report?
The report says GPT-6 Astra identified the games behind four small networks, found copying heads in three small language models, and generated networks that beat trained comparisons in three games. These findings have not been independently verified in the supplied material.
Some narrow measures improved, while another trial produced no useful result. One local model completed more legal answers, but its legal score remained zero; a JSON test score rose from 49 to 70 out of 120.
Does this mean AI models can now be reliably repaired by editing weights?
No. The report presents early experiments and limited personal tests, not evidence of reliable repairs across models or tasks. Independent replication and broader evaluations are still needed.
Could direct weight edits affect a model’s safety behavior?
The report says the same kind of edit could potentially strip safety training from a model. It presents this as a risk, not as a demonstrated outcome in the experiments described.
Source: ThorstenMeyerAI.com
