🔍 Read the full analysis: From 49 To 70 Of 120: GPT-6 Astra’s Qwen Weight Edit on ThorstenMeyerAI.com
Get smart everyday buys delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
An internal experiment by GPT-6 Astra, running as a Codex agent, changed one matrix in Qwen3-4B-Instruct-2507. On a frozen synthetic test of copying special-character text into JSON, exact answers rose from 49 of 120 to 70, with 21 cases fixed and none made wrong; table-row extraction remained at zero of 30.
GPT-6 Astra, operating as a Codex agent, increased a Qwen model’s exact-answer score from 49 to 70 of 120 cases in an internal test by editing one weight matrix. The result applies to a narrow synthetic task—preserving tabs and other special characters when copying text into JSON—and does not show a broader improvement in the model’s abilities.
The experiment used Qwen3-4B-Instruct-2507, an open model described in the source report as licensed under Apache-2.0. The agent calculated a rank-one update to the output projection matrix in the model’s final multilayer perceptron layer. The report says there was no gradient training and that the other 397 stored tensors remained bit-identical to the original.
On the frozen final test, the edited model produced 70 exact answers out of 120, compared with 49 for the unedited model: a gain of 21 cases, or 17.5 percentage points. The report says no previously correct case became incorrect. Exactness required the returned JSON to decode to the original text character for character; for example, a tab had to be represented as the JSON escape sequence \t.
The test was built around a weakness identified in an earlier attempt. In that attempt, both model versions returned 47 invalid answers, mostly by placing literal tab characters inside JSON strings instead of escaping them. Before the weight edit, a screening set produced 16 exact answers out of 32. A more explicit prompt also scored 16 of 32, according to the report.
Infographic 01 · The measured result
Where the edit helped, and where it did not
Qwen3-4B-Instruct-2507, original versus one edited matrix, on synthetic texts that must survive the trip into JSON without losing a character.
Final test · 120 cases by group
- Texts with tabsthe target failureOriginal19/30Edit30/30
- Line breaksalready perfectOriginal30/30Edit30/30
- Mixed special characterstabs, quotes, backslashesOriginal0/30Edit10/30
- Second table rowstill unsolvedOriginal0/30Edit0/30
Development, validation and controls
- Developmentused to fit the edit · not independentOriginal24/40Edit40/40
- Validationnew cases · 9 wins, 0 lossesOriginal10/40Edit19/40
- Random directioncontrol on validation · same strengthOriginal10/40Edit10/40
- Plain-text tabscontrol · real tabs must surviveOriginal10/10Edit10/10
Exact answer = after normal JSON parsing, the text field matches the source text character for character. Same prompt, temperature 0, seed and output limit for both models; saved and reloaded edit, no hooks, no grammar constraints, no repair. Final test, validation and controls frozen before any independent output of the edited model. Fifty of 120 final answers are still wrong.
Sources: our experiment records, 5 Nov 2026 (Lexassay research, internal): experiment 3 result report and final result by GPT-6 Astra working as a Codex agent (OpenAI). Synthetic test cases, not published.
Powered by Thorsten Meyer AI
A Narrow Gain, Not a General Fix
The result suggests that a targeted change to a model’s stored weights can improve performance on a specific, previously observed failure mode—even when a stronger prompt did not improve the screening score. That makes the experiment a useful data point in work on editing neural networks directly, rather than relying only on prompts or conventional training.
Its practical reach remains limited. Fifty of 120 final cases were still wrong, and the report says a separate table-row extraction task remained at 0 of 30. The result therefore does not establish that the edit improves JSON handling generally, or that the modified model is more reliable across unrelated tasks. The test cases were synthetic, and no independent review of this experiment had been completed when the report was published.
Products Worth Considering
Apple MacBook Pro MPHF3LL/A 14.2-Inch Laptop | M2 Pro 12-Core Chip 16GB Unified Memory 1TB SSD | 19-Core GPU Liquid Retina XDR Display Space
Apple MacBook Pro 2023 16.2 M2 Max 32GB 1TB SSD Space Gray
HP Sprocket Portable 2×3 Instant Photo Printer, Lilac, with ZINK Paper Bundle
As an affiliate, we earn on qualifying purchases.
How the Three Experiments Led Here
The report places this work in a week-long series of experiments involving AI models and the weights of other models. It says an earlier experiment used Claude to edit a local model called Ornith. That model completed more legal-answer prompts after the edit, but its legal score remained zero. GPT-6 Astra later reviewed that work and identified issues including a resume bug, a command-line problem and side effects from weight conversion.
Before the current experiment, GPT-6 Astra also tried editing Qwen for a different task: extracting five fields from synthetic business messages. The final test scores were 97 of 200 for both versions. The report says that attempt exposed repeated invalid JSON outputs caused by raw tabs, giving the next experiment a specific failure to target rather than a broad search for model weaknesses.
For the present test, the site owner instructed the agent on November 5 to inspect Qwen, find a weakness and attempt an improvement. The report says GPT-6 Astra planned the experiment, created and froze the tests, made the matrix edit and measured the outcome. It also says Claude did not participate in the experiment itself; a Claude agent later researched and drafted the article and checked the recorded figures.
Independent Checks Still Pending
The reported score is based on the experiment records described by the source, but the material does not provide an independent replication or a second AI review of this edit. The tests are synthetic, and the source does not establish how the change would perform on real-world JSON inputs or other tasks.
The available account also does not give enough detail to assess the full test-generation process, the range of possible inputs, or whether the improvement persists across repeated runs and different model settings. The reported 21-case gain should be read as the outcome of this particular frozen test, not as a measured general improvement in Qwen’s capabilities.
Replication and Broader Testing
The source article identifies an independent review as the missing check, particularly because a prior experiment in the series received scrutiny that uncovered faults. No date or plan for that review is given. A useful next step would be to repeat the frozen test independently and report results on additional inputs, while checking whether the modified weights cause regressions outside the target task.
Until such checks are reported, the finding remains a single internal experiment with a measurable but task-specific result. The source does not announce a release of the edited weights or a deployment plan.
Key Questions
What did the weight edit change?
The report says GPT-6 Astra applied a rank-one update to one output-projection matrix in Qwen3-4B-Instruct-2507’s final MLP layer. It reports no gradient training and says the other 397 stored tensors were unchanged.
What does “49 to 70 of 120” measure?
It is the number of cases answered exactly on a frozen synthetic test. The task required copying text with special characters into JSON so that decoding the JSON reproduced the original text without any character changes.
Did the edit improve every tested task?
No. The reported gain was for the special-character JSON copying test. Table-row extraction remained at 0 of 30, and 50 of the 120 final test answers were still incorrect.
Has another reviewer verified the result?
Not according to the source article, which says this experiment had not yet received an independent review. The reported figures come from the experiment records described there.
Source: ThorstenMeyerAI.com
