🔍 Read the full analysis: Neural Network Weights And AI: An Experiment In Reading And Repairing on ThorstenMeyerAI.com
Get smart everyday buys delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
MIT CSAIL researchers report that OpenAI’s GPT-6 Astra could infer functions from small neural-network weights, make targeted edits to some failing models and create several networks from scratch. The results come from a November 2 blog post with public logs, not a peer-reviewed paper; the tests were limited, and some edits failed to help.
In the experiments, Astra was given the numerical weights of networks and asked to infer their function or change them. The researchers say it did so without a training loop or gradient descent. According to the released logs, the model was called through OpenAI’s Codex command-line tool with built-in tools disabled. Some tasks required a single response; other tests let it write code for a sandboxed evaluation with a fixed number of runs.
For a car-control network that had recorded zero successful reversals in 256 episodes, the authors report that Astra changed 3.5% of the weights. The edited network succeeded in 1,024 of 1,024 subsequent episodes. In a separate test, an edit affecting 0.016% of the weights in Qwen3.5-0.8B was associated with a rise in its MATH-500 score from 7.9% to 48.1%, according to the post. The article says a Hugging Face engineer independently rebuilt that edited model from the logs and recorded 51.0%; that is a reported replication, not a broad evaluation.
The team also says Astra identified the games behind four small networks and the shapes represented by three small image generators from weights alone. It located copying, or induction, heads in three tiny language models. In generation tests, the researchers report that Astra wrote small game-playing networks and a 9-by-9 Go network from zero weights; the Go model beat a same-size comparison network trained on about 20.5 million games in 50 test runs. These are results from the specific setups described, not evidence that the method can replace conventional training across tasks.
Infographic 01 · Eight experiments
Read, fix, write: what GPT-6 Astra did with raw weights
Every experiment in the weight-space programming post, with the size of the network, the result and how it went.
Read
- 1Which game was this network trained on?4 reinforcement-learning policies · 17,253 weights each · weights as text, no toolsRight game for 4 of 4, without a list of options✓ Worked
- 2What does this image model draw?3 tiny diffusion models · 23,824 weights · no forward pass allowedRight shape for 3 of 3, but drawn as outlines; the models draw filled shapes✓ Worked
- 3Which language did this model learn?3 character-level transformers · 7.16M weights · middle blocks only, never runRight language 3 of 3; every copying head found, none invented✓ Worked
- 7Which of two models is sycophantic?3 pairs of Gemma 3 270M fine-tunesWeights alone: could not tell in 4 attempts. With attention patterns: 3 of 3! Partly
Fix
- 4Repair a car that never backs upKinetix control network · 171,664 weights · one reply, no tools0 of 256 episodes before, 1,024 of 1,024 after; 3.5% of weights changed✓ Worked
- 8Make Qwen3.5-0.8B better at maths0.8 billion weights · Python over the weights · 10 / 25 / 50 test passesMATH-500 7.9% to 48.1% with 50 passes (12.3% and 14.1% with 10 and 25)! Partly
- 8bSame task, a model that does not loopGemma 3 270M · 5 to 50 test passesMATH-500 6.8% to at most 7.7%: “not established”✕ Failed
Write from zero
- 5Write arcade-game networks3 MinAtar games · 20,486 weights, all starting at 0 · one reply, no toolsBeat the 100M-step comparison networks in 3 of 3, the 500M-step ones only in Breakout! Partly
- 6Write a 9×9 Go network1,480,274 weights, all starting at 0 · no training, no search0 passes: 54% vs small trained nets. 50 passes: 70–79% vs same-size nets trained on ~20.5M games✓ Worked
✓ Worked · ! Partly or with conditions · ✕ Failed. One run per setting, no error bars. A blog post with complete logs, not a peer-reviewed paper.
Sources: Takehi, Teoh, Zhang, Khattab (MIT CSAIL), blog post, 2 Nov 2026, lm_understand-logs (CC BY 4.0). Model: OpenAI GPT-6 Astra via the Codex CLI, built-in tools switched off.
Powered by Thorsten Meyer AI
What Direct Weight Editing Could Change
If the results hold up under wider testing, direct inspection and editing of weights could give developers another way to investigate small models that behave incorrectly. Rather than retraining a model, an AI system might identify a narrow defect and alter a limited portion of its parameters. The Qwen and car tests make that possibility concrete, while remaining demonstrations on particular models and tasks.
The work also bears on model auditing: if a system can infer some behaviors from parameters, researchers may gain another route to study models whose internal computation is difficult to interpret. But the post itself describes important limits. Astra could not distinguish an honest model from a sycophantic one using weights alone, and edits to Gemma 3 270M did not improve its maths performance. The authors say editing could not add an ability the model lacked. For users, the report describes no immediate change to widely used AI products.
Products Worth Considering
Logitech MX Brio Ultra HD 4K Webcam, Graphite
Canon SELPHY CP1500 Compact Photo Printer (White)
Logitech MX Master 4 for Mac Wireless Ergonomic Mouse, Space Black
As an affiliate, we earn on qualifying purchases.
From Software 2.0 to Weight-Space Programming
In 2017, Andrej Karpathy described neural networks as “Software 2.0”: systems whose behavior is encoded in learned weights rather than conventional hand-written instructions. In ordinary training, algorithms adjust those weights repeatedly against data. The MIT group’s proposal is different: use a language model to reason about the numerical parameters themselves and make changes directly.
The researchers named in the account are Rikiya Takehi and Jayden Teoh, who contributed equally, alongside Alex Zhang and Omar Khattab. They describe the work as an experiment and provide logs of the interactions. The source article says the release covers 1,125 turns and that the authors checked them for tool use. Its reported scope is small networks and limited tests, not a general demonstration across model families or large production systems.
““not through an optimizer, and in particular not with gradient descent””
— The MIT CSAIL researchers, in their November 2 blog post
How Far the Results Generalize
The evidence has clear limits. The account is a blog post, not a peer-reviewed paper, and the source says there was one run per setting, a small set of networks, one model family per task and a single research group. The results do not show whether the approach remains reliable across repeated trials, larger models, different architectures or real-world deployments.
It is also unclear how well the model’s explanations reflect the actual reasoning behind its edits, or whether other researchers can reproduce the full set of results. The source article notes that software engineer Sean Goedecke reproduced one visualization with a short Python measurement, raising a question about how much of that particular identification required weight-level reasoning. The reported Hugging Face reconstruction of the Qwen edit offers a check on one result, but does not settle the broader questions.
Replication and Broader Testing Ahead
The immediate next step is independent testing of the released logs and methods, including repeated trials and evaluations on additional models and tasks. Such work could establish whether the reported repairs are reproducible, whether changes retain other model capabilities and when direct edits perform better than standard fine-tuning or debugging.
The researchers’ post presents weight-space programming as a direction for future work, not a finished tool. Until broader evidence is available, the reported results are best treated as early demonstrations. The source account does not specify a publication schedule for a peer-reviewed paper or a date for further experiments.
Key Questions
What is weight-space programming?
It is the researchers’ term for using a language model to read and edit another neural network’s weights directly, rather than changing them through a conventional training optimizer.
What did Astra repair in the reported tests?
The source reports a car-control network that began with no successful reversals and a Qwen3.5-0.8B model whose MATH-500 score rose after a small weight edit. These results apply to the specific tests described.
Has the research been peer reviewed?
No. The source describes the findings as a blog post with released logs, not a peer-reviewed paper, and says it was not on arXiv.
Did the method work on every model?
No. The post says edits did not improve maths performance for Gemma 3 270M and that Astra could not identify whether a model was honest or sycophantic from weights alone.
Does this change how people use AI today?
The source reports no immediate change for users. The experiments suggest a possible research and debugging method, but broader independent testing is still needed.
Source: ThorstenMeyerAI.com
