refactor(skills): fold optimize into autotune and test-fix into test, stop questions in autonomous runs

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Kevin Veen-BirkenbachandClaude Opus 5.5 committed 2026-10-30 16:54:37 +01:00
1 parent 20a47e3916
commit 1ac809f884
11 files changed
+120 -171

No files matched your search

+3 -3
View File
@@ -7,9 +7,9 @@ description: >
and the pattern is general enough to reuse. Portable across projects.
---
Its objective is the `optimize` skill's: a skill is worth creating only when it
cuts tokens or time and raises quality — judge every candidate against that
skill's three axes and rules.
Its objective is the `autotune` skill's **Objective** section: a skill is worth
creating only when it cuts tokens or time and raises quality; judge every
candidate against its three axes and rules.
When you notice the operator repeating the same kind of instruction,
correction, or prompt pattern (about three occurrences, exact wording may
+30 -4
View File
@@ -13,9 +13,35 @@ Autotune turns observed friction into durable skills. It runs on demand only,
it changes nothing without an explicit answer from the operator, and every
skill it writes or rewrites lands in the operator's skills repository.
Its objective is the `optimize` skill's: every proposal must cut tokens or time
and raise quality — follow that skill's three axes and rules, and drop any
candidate that moves none of them.
## Objective
The shared objective of the self-improving skills (autotune, autoskill,
confidence): every skill, rule, or report the agent produces about its own
working is judged on three axes at once. A proposal that cannot name the axis
it moves is not an optimization, it is a preference.
- **Tokens** — fewer input and output tokens for the same result: read the range
instead of the whole file, grep the saved log instead of re-running the
command, route to an authoritative doc instead of duplicating it, report in
lines instead of paragraphs.
- **Time** — fewer round trips and less waiting: independent calls in parallel,
one command that answers the question instead of three that circle it, no
polling for work the harness will report on its own.
- **Quality** — fewer errors and less rework: verification instead of
assumption, and learnings that persist so the same mistake is not paid for
twice.
- A change must improve at least one axis and damage none. Output that got
cheaper but lost correctness is a regression, not an optimization.
- Quality is never the currency: validation, error handling, tests, and security
steps are not what gets cut to save tokens or time.
- Measure, do not guess: name the observed friction — the re-read, the retry,
the correction the operator had to give twice — and what it cost. A guessed
percentage is not a measurement.
- The cheapest step is the one that does not run: drop work before optimizing
it.
- Persist it or pay again: an optimization that lives only in this session is
re-derived in the next one, so it belongs in a skill or a memory entry.
## Trigger discipline
@@ -119,6 +145,6 @@ reverse — remove the mirrors too, or the old name keeps firing.
whole collection is not what autotune is for.
- Never write a skill for one-off work, for secrets, or for behaviour an
existing skill already covers - propose extending that skill instead.
- The `optimize` skill's rules decide what counts as an improvement: at least
- The **Objective** section decides what counts as an improvement: at least
one axis better, none damaged, and the claim named as observed friction rather
than a guessed percentage.
+2 -6
View File
@@ -35,12 +35,8 @@ added lines only) and delete every explanatory comment:
both the lint and the language parser can pass over code that no longer
executes. Diff against the pre-edit revision (`git show HEAD:<path>`) and
confirm that only comment lines left.
5. Do not stage in order to run a check. Repository lints that enumerate through
`git ls-files` already see every tracked file and read its worktree content;
only an untracked NEW file is invisible to them and needs `git add`. Staging
a tracked, modified file before a lint changes nothing except the operator's
index, which they then have to undo. Stage only what a requested bundle
contains, and re-stage after the audit exactly the files that were staged
5. Do not stage in order to run a check (the `commit-next` skill's step 4
says why). Re-stage after the audit exactly the files that were staged
before it.
6. Penalty: if this audit finds anything you wrote yourself, admit it in
the report (file and count, no excuses), lower your confidence numbers
+1 -1
View File
@@ -37,5 +37,5 @@ A threshold the operator named ("95% confidence, break risk ≤5%") is a gate on
the work, handled as the `break-risk` skill's last section says, for both
numbers.
Objective: the `optimize` skill's. Two figures with their residuals replace a
Objective: the `autotune` skill's **Objective** section. Two figures with their residuals replace a
hedging paragraph and save the operator a verification round trip.
+4
View File
@@ -19,6 +19,10 @@ needs. A dialectic run on the wrong question wastes the whole loop. Return to
`active-listening` mid-loop whenever an objection turns on a fact only the
operator holds.
When another skill or an autonomous run (robot, observe, a loop) invoked this
skill, derive the requirements from the task and the evidence instead of
asking; ask only when an operator-only fact blocks the conclusion.
## The loop
1. **Thesis.** Form the strongest initial claim you can, grounded in evidence,
-40
View File
@@ -1,40 +0,0 @@
---
name: optimize
description: >
The shared objective of the self-improving skills: raise efficiency — fewer
tokens, less time — and raise quality at the same time. Trigger from autotune,
autoskill and confidence, which route their objective here, and on /optimize
when the operator asks how to make the agent's own work cheaper, faster or
better. Portable across projects.
---
Every skill, rule, or report the agent produces about its own working is judged
on three axes at once. A proposal that cannot name the axis it moves is not an
optimization, it is a preference.
## The three axes
- **Tokens** — fewer input and output tokens for the same result: read the range
instead of the whole file, grep the saved log instead of re-running the
command, route to an authoritative doc instead of duplicating it, report in
lines instead of paragraphs.
- **Time** — fewer round trips and less waiting: independent calls in parallel,
one command that answers the question instead of three that circle it, no
polling for work the harness will report on its own.
- **Quality** — fewer errors and less rework: verification instead of
assumption, and learnings that persist so the same mistake is not paid for
twice.
## Rules
- A change must improve at least one axis and damage none. Output that got
cheaper but lost correctness is a regression, not an optimization.
- Quality is never the currency: validation, error handling, tests, and security
steps are not what gets cut to save tokens or time.
- Measure, do not guess: name the observed friction — the re-read, the retry,
the correction the operator had to give twice — and what it cost. A guessed
percentage is not a measurement.
- The cheapest step is the one that does not run: drop work before optimizing
it.
- Persist it or pay again: an optimization that lives only in this session is
re-derived in the next one, so it belongs in a skill or a memory entry.
-95
View File
@@ -1,95 +0,0 @@
---
name: test-fix
description: >
Run the project's Makefile test targets in a loop - run, diagnose, fix, re-run -
until every selected target is green or the loop is provably stuck. Trigger on
/test-fix or when the operator asks to make the tests pass, fix the failing
tests, or get the suite green. Portable across projects.
---
Same target discovery and selection as the `test` skill - invoke it for step 1
and step 2 rather than reimplementing the grep and the selection list. This
skill owns what happens after the first red result: the fix loop.
The contract is narrow: the code becomes correct, the tests stay honest.
## The loop
One iteration is: run -> read the failure -> one root cause -> one fix ->
re-run. Never batch several speculative fixes into one iteration; a green run
after three simultaneous changes proves nothing about which one mattered.
### Run
Run the failing target alone, not the whole selection - the fastest command
that reproduces the failure. Widen back to the full selection only when that
target is green.
### Diagnose
Read the actual error output before touching a file. Apply the `triage` skill
to reach a verified root cause; for a failure whose cause is genuinely unclear
after one look, apply `dialectic` rather than guessing twice.
Decide explicitly which side is wrong, and say so in one line before editing:
- **Code is wrong**: the test states the intended behaviour. Fix the code.
- **Test is wrong**: the test encodes an outdated or incorrect expectation.
Fixing it is allowed *only* with a stated reason for why the old expectation
was wrong - never because the code disagrees with it.
- **Environment is wrong**: missing dependency, stale container, absent env var,
unbuilt artefact. Fix the environment or report it; do not patch code around
a broken environment.
### Fix
Smallest change that addresses the root cause. Follow the project's existing
idioms and the `no-defaults` and `comments-clean` rules.
Forbidden, in every iteration, without explicit operator approval:
- deleting, skipping, `xfail`-ing, or commenting out a failing test
- loosening an assertion to whatever the code currently produces
- catching or swallowing the exception the test was written to surface
- adding retries, sleeps, or reruns to paper over a flaky failure
- disabling a linter rule, a type check, or a test target from the selection
Any of these is a report item, not a fix. A suite that is green because the
failing test no longer runs is a regression disguised as success.
### Re-run
Re-run the same target. Then, once it passes, re-run every target that was
already green - a fix that breaks a neighbouring suite is not a fix. Only
after the full selection is green is the loop done.
## Stopping
Stop and report, without asking for permission to stop:
- **Green**: full selection passes. Report and stop.
- **No progress**: the same failure survives 3 iterations, or the failure count
stops falling across 3 iterations. Report the root cause reached so far, what
was tried, and why each attempt failed.
- **Fix exceeds the mandate**: the real fix is an API change, a dependency bump,
a schema migration, or a redesign. Name it, show the minimal diff it would
need, and let the operator decide.
- **Oscillation**: fixing A re-breaks B and vice versa. Report both with the
conflict between them - that is a design problem, not a test problem.
- **Flaky**: a target passes and fails without any change between runs. Report
it as flaky with both outputs; never "fix" it by rerunning until green.
Never loop silently. Emit one line per iteration: target, failure, hypothesis,
change made.
## Report
- Per iteration: what failed, the root cause, the fix, the result.
- Final state of every selected target, pass or fail, with the verbatim output
of anything still failing.
- Everything deliberately not done: tests judged wrong but left alone,
environment issues, out-of-mandate fixes.
- The exact command reproducing the final state.
State calibrated confidence in the fixes per the `confidence` skill. Do not
commit unless the operator asked for a commit.
+60 -5
View File
@@ -2,9 +2,10 @@
name: test
description: >
Discover the test targets of the project's Makefile, let the operator pick
which ones to run, then run them and report. Trigger on /test or when the
operator asks to run the tests, the test suite, or a specific test target
without naming the exact command. Portable across projects.
which ones to run, then run them and report; in fix mode, loop run, diagnose,
fix, re-run until green. Trigger on /test, /test --fix, or when the operator
asks to run the tests, make them pass, or get the suite green. Portable
across projects.
---
The Makefile is the source of truth for how this project runs tests. Never
@@ -73,5 +74,59 @@ Per target: pass/fail plus the failing test names and the exact error output,
quoted verbatim. Never paraphrase a failure. Then one line with the exact
command to reproduce the failure alone.
Do not fix what failed unless the operator asks - report first. If the operator
does ask for a fix, apply the `triage` skill to it.
Do not fix what failed unless the operator asks - report first. A request to
make the tests pass, fix them, or get the suite green (`/test --fix`) switches
to fix mode.
## Fix mode
The contract is narrow: the code becomes correct, the tests stay honest.
One iteration is: run -> read the failure -> one root cause -> one fix ->
re-run. Never batch several speculative fixes into one iteration; a green run
after three simultaneous changes proves nothing about which one mattered.
- **Run** the failing target alone, the fastest command that reproduces the
failure. Widen back to the full selection only when that target is green.
- **Diagnose** from the actual error output before touching a file. Apply the
`triage` skill to reach a verified root cause; for a cause still unclear
after one look, apply `dialectic` rather than guessing twice. Say in one line
which side is wrong:
- **Code**: the test states the intended behaviour. Fix the code.
- **Test**: it encodes an outdated or incorrect expectation. Fix it only with
a stated reason why the old expectation was wrong, never because the code
disagrees with it.
- **Environment**: missing dependency, stale container, absent env var,
unbuilt artefact. Fix the environment or report it; do not patch code
around it.
- **Fix** with the smallest change that addresses the root cause, in the
project's idioms and under the `no-defaults` and `comments-clean` rules.
Forbidden without explicit operator approval: deleting, skipping, `xfail`-ing
or commenting out a failing test; loosening an assertion to what the code
produces; swallowing the exception the test surfaces; retries, sleeps or
reruns over a flaky failure; disabling a linter rule, a type check or a
target. Any of these is a report item, not a fix: a suite that is green
because the failing test no longer runs is a regression disguised as success.
- **Re-run** the same target, then every target that was already green; only
a full green selection ends the loop.
Stop and report, without asking for permission to stop:
- **Green**: the full selection passes.
- **No progress**: the same failure survives 3 iterations, or the failure count
stops falling across 3 iterations. Report the root cause reached, what was
tried, and why each attempt failed.
- **Beyond the mandate**: the real fix is an API change, a dependency bump, a
schema migration, or a redesign. Name it with its minimal diff and let the
operator decide.
- **Oscillation**: fixing A re-breaks B and vice versa. Report both; that is a
design problem, not a test problem.
- **Flaky**: a target passes and fails without any change. Report both
outputs; never rerun until green.
Emit one line per iteration: target, failure, hypothesis, change. The final
report gives per iteration what failed, the root cause, the fix and the
result; the final state of every selected target with the verbatim output of
anything still failing; everything deliberately not done; and the command
reproducing the final state. State confidence per the `confidence` skill. Do
not commit unless the operator asked for a commit.
+8 -11
View File
@@ -3,10 +3,9 @@ name: triage
description: >
Triage a failing CI run, pipeline, or job set to a verified root cause and a
real fix. Trigger when a build, test, or deploy pipeline has one or more failed
jobs. Applies the dialectic skill to every failing job independently, the
democracy peer protocol to every failure cluster, and the funnel skill to every
cluster, so each cluster is fixed in its own worktree, and hands one log per
cluster to the tune skill. Portable across projects.
jobs. Applies the dialectic skill, the democracy peer protocol and the funnel
skill to every failure cluster, so each cluster is fixed in its own worktree,
and hands a timed-out job's log to the tune skill. Portable across projects.
---
Drive a red pipeline to green by root cause, not by guesswork. Treat every
@@ -82,13 +81,11 @@ jobs' logs, and a first clustering. The judgement in those steps stays yours.
honestly instead of fixing around it. Report every fix, every proposal, and
every abandoned attempt to the same agents through the `democracy` Peer protocol,
then remove the sidetree.
9. **Tune one log per cluster.** For each cluster, hand one job log to the
`tune` skill: the cluster's job that ran furthest, because its timings cover
the most phases. `tune` looks for performance potential in it under its own
rules and the session's limits. Its candidates run through funnels of their
own, separate from the cluster's fix sidetree, and never delay or block the
fix. A log the failure cut short is tuned for the phases it covers; say which
phases it does not reach.
9. **Tune what ran out of time.** Only for a cluster whose jobs died on their
time budget (timed out, or cancelled by the runner's limit), or when the
operator asks: hand the job log that ran furthest to the `tune` skill. Its
candidates run through funnels of their own, separate from the cluster's fix
sidetree, and never delay or block the fix.
10. **Follow the run.** While the run is still in progress, re-check it
periodically and triage each newly failed job as it appears. The run is done
only when it has finished and every failure has a verified fix.
+4
View File
@@ -42,6 +42,10 @@ suggestions, and allow free text:
Keep the answers for the rest of the session and do not ask again. Work only
inside them; a candidate outside them becomes a proposal.
When another skill (triage, observe) or an autonomous run invoked tune, skip
the questions: examine the handed-over log, apply the defaults above, and ask
only when an operator-only fact blocks.
## Forbidden, whatever it saves
- Dropping a test, a spec, a play, a drill or a verification pass.
+8 -6
View File
@@ -1,4 +1,4 @@
"""The self-improving skills must share the optimize objective."""
"""The self-improving skills must share the objective held by autotune."""
from __future__ import annotations
@@ -7,13 +7,13 @@ from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parents[1]
SKILLS = REPO_ROOT / "skills"
ROUTERS = ("autotune", "autoskill", "confidence")
ROUTERS = ("autoskill", "confidence")
class TestOptimizeObjective(unittest.TestCase):
def test_skill_names_the_three_axes(self):
text = (SKILLS / "optimize" / "SKILL.md").read_text(encoding="utf-8")
self.assertIn("name: optimize", text)
def test_autotune_names_the_three_axes(self):
text = (SKILLS / "autotune" / "SKILL.md").read_text(encoding="utf-8")
self.assertIn("## Objective", text)
for axis in ("**Tokens**", "**Time**", "**Quality**"):
self.assertIn(axis, text)
@@ -21,7 +21,9 @@ class TestOptimizeObjective(unittest.TestCase):
for skill in ROUTERS:
text = (SKILLS / skill / "SKILL.md").read_text(encoding="utf-8")
self.assertIn(
"`optimize` skill", text, f"{skill} does not route to optimize"
"`autotune` skill's **Objective**",
text,
f"{skill} does not route to the autotune objective",
)