Before yesterday’s fix becomes today’s rule
Reading about self-evolving harnesses made me more curious about keeping a revision than about generating one.
A line follows the next task
If a rule written to prevent a mistake gets in the way of the next task, where should the correction happen?
Self-Evolving Agent Harness examines revisions to prompts, memory, workflows, and executable scaffolding while model weights remain fixed. Accepted changes persist into later runs. It is an October 2026 preprint.
The persistence caught my attention. Fixing this answer can end on this screen. Editing shared instructions sends a line into tasks we have not encountered yet.
Suppose a document summary omits a deadline. We add “preserve every date” to prevent a repeat. Give it the failed document again, and the deadline appears. That is reassuring.
Next, we ask for a short summary of an old record. Dates from the past now trail through it. The earlier omission is fixed, while the later request becomes harder to read.
One failed document has become the editor of the shared instructions.
Between proposing and keeping
The paper separates generating a revision from admitting it. I like that distinction. Producing a plausible edit need not make it the next run’s default.
For the hypothetical summary rule, I would first record what we meant to repair: preserve the deadline. Keeping every historical date was not the original aim.
Then there is more to read than the document that failed. Previously satisfactory summaries, date-heavy records, and writing without schedules all offer different questions. We also need tasks that were not used to devise the rule.
There is no basis for expecting every result to improve. We could adopt the revision, narrow its scope, or leave it as a correction to one document. An edited line does not have to graduate into a shared rule.
This is an operating approach I am considering after reading. The paper’s reliability analysis rests on explicit assumptions. I would not read it as confirmation that automatically edited instructions improve every task.
If the scorecard changes too
Another detail stayed with me: the paper keeps the final assessment outside what revision candidates may change. It draws a boundary between provisional checks and the eventual judgment of the result.
After changing the rule to retain deadlines, changing the evaluator to reward more dates would make improvement easy to claim. It would make answering the original request harder to assess. This, too, is hypothetical.
Naresh Jain’s separate coding-agent experiment also distinguishes feedback during development from external evaluation. Its inner harness did not receive the outer evaluator’s answer key. I have not reproduced that experiment, and results from one service cannot establish the effect of every harness.
Keeping revision and judgment distinct leaves a place to compare. Liking the new rule and observing better adherence to the same requirements can go into separate records.
Even with the same model, changed instructions and tools alter the conditions of a run. I would therefore record the harness version beside the model name. If a later answer differs from an earlier one, there should be somewhere to look back.
A note not yet attached
If I maintained this hypothetical summarizer, I would keep the old instructions beside the proposed edit, with the omission that prompted it and the tasks where it helped or got in the way.
A growing record need not all become instruction text. The place for recording failures can differ from the rules passed to the next task.
I picture a small sheet on the desk saying “preserve every date.” Before attaching it to the shared instructions, I put an unread document beside it.
Would that line help here too? Moving it into the common rules can wait until after that question.