feat(check): score fix, break, efficiency and speed in one block
🧪 Test / 🧪 Lock + lint (push) Waiting to run
🧪 Test / 🧪 Lock + lint (push) Waiting to run
kvb-check-efficiency measures a change against its base: tokens.py counts skill text tokens and listing words, measure.py runs one command in a base worktree and the working tree (one run per side, the mean of five over 100, an estimate marked ~ when other processes load the CPU). kvb-check-speed reads wall time from the same run and is not relevant for skills. kvb-check-score prints fix, break, efficiency, speed and the product score with emojis plus a gate line: below 90/5 it reports, over 100 it requires asking the operator before the commit. Every skill that reported confidence and break risk now shows the score block; the 90/5 bar adds efficiency and speed at most 100. Commit blocks and the commit call's approval prompt name worktree, branch, diff stat and the score values, also for merges. 24 descriptions were shortened to keep the skill listing within its budget. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
6fa0311704
commit
0343a2f3ce
46 files changed
+1145
-118
No files matched your search
@@ -64,6 +64,9 @@ flowchart LR
|
||||
check --> check_confidence[kvb-check-confidence]
|
||||
check --> check_break_risk[kvb-check-break-risk]
|
||||
check --> check_gate[kvb-check-gate]
|
||||
check --> check_efficiency[kvb-check-efficiency]
|
||||
check --> check_speed[kvb-check-speed]
|
||||
check --> check_score[kvb-check-score]
|
||||
org --> org_organize[kvb-org-organize]
|
||||
org --> org_split[kvb-org-split]
|
||||
org --> org_coordinate[kvb-org-coordinate]
|
||||
@@ -161,6 +164,9 @@ flowchart LR
|
||||
confidence[kvb-check-confidence]
|
||||
break_risk[kvb-check-break-risk]
|
||||
gate[kvb-check-gate]
|
||||
efficiency[kvb-check-efficiency]
|
||||
speed[kvb-check-speed]
|
||||
score[kvb-check-score]
|
||||
end
|
||||
subgraph cat_org [org]
|
||||
democracy[kvb-org-organize]
|
||||
@@ -237,6 +243,13 @@ flowchart LR
|
||||
confidence --> break_risk
|
||||
confidence --> gate
|
||||
pr_label --> gate
|
||||
stage_req -.-> score
|
||||
score --> confidence
|
||||
score --> break_risk
|
||||
score --> efficiency
|
||||
score --> speed
|
||||
speed --> efficiency
|
||||
efficiency -. heavy commands .-> queue
|
||||
gate --> democracy
|
||||
gate --> hotfix
|
||||
triage --> hotfix
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
---
|
||||
name: kvb-check-break-risk
|
||||
description: >
|
||||
Derive the break-risk percentage from measurement, not feel. Called by
|
||||
`kvb-check-confidence` on every report; also when the operator asks "what could this
|
||||
Derive the break-risk percentage from measurement, not feel, for every
|
||||
`kvb-check-confidence` report; also when the operator asks "what could this
|
||||
break?" or names a break-risk threshold.
|
||||
---
|
||||
|
||||
@@ -43,7 +43,8 @@ Read it when a step is new to you or its result is contested.
|
||||
## Producing the number
|
||||
|
||||
State it as a percentage next to fix confidence, never blended, and always with
|
||||
its residuals. Anchor roughly:
|
||||
its residuals; `kvb-check-confidence` passes it to the `kvb-check-score` block
|
||||
as `--break`. Anchor roughly:
|
||||
|
||||
- **Under 5%**: consumers enumerated, delta measured at the surface, every axis
|
||||
of the matrix covered by executed evidence, failure mode closed.
|
||||
|
||||
@@ -2,22 +2,28 @@
|
||||
name: kvb-check-confidence
|
||||
description: >
|
||||
Report calibrated fix confidence and break risk unprompted whenever you
|
||||
stage changes, declare a fix or bundle ready, or summarize work the operator
|
||||
may commit or deploy.
|
||||
stage changes, declare a fix or bundle ready, or summarize committable or
|
||||
deployable work.
|
||||
---
|
||||
|
||||
Every stage summary, fix report, or ready-to-commit bundle ends with a
|
||||
confidence block. Invoke the `kvb-check-break-risk` skill every time before
|
||||
writing it; this skill reports the number it derives and never estimates one
|
||||
itself.
|
||||
Every stage summary, fix report, or ready-to-commit bundle ends with the
|
||||
score block of the `kvb-check-score` skill. Invoke the `kvb-check-break-risk`
|
||||
skill every time before writing it; this skill reports the number it derives
|
||||
and never estimates one itself.
|
||||
|
||||
1. **Fix confidence** (percent): probability the change does what it claims.
|
||||
**Break risk** (percent): probability it breaks anything else, from the
|
||||
`kvb-check-break-risk` skill. Two numbers, never one blended figure.
|
||||
2. **Evidence**: the strongest proof lines only: what was executed,
|
||||
2. **Score**: pass both, with efficiency from the `kvb-check-efficiency`
|
||||
skill and speed from the `kvb-check-speed` skill, to
|
||||
`${CLAUDE_SKILL_DIR}/../kvb-check-score/scripts/score.py --fix <n> --break
|
||||
<n> --efficiency <n|-> --speed <n|->` and show its score line and gate line
|
||||
verbatim. Exit 1 fails the 90/5 rule; exit 3 means ask the operator again
|
||||
before the commit, naming the cause the gate line prints.
|
||||
3. **Evidence**: the strongest proof lines only: what was executed,
|
||||
reproduced, enumerated, or read from an authoritative source. Mark proven
|
||||
(ran it, saw it) apart from derived (reasoned statically).
|
||||
3. **Residuals**: exactly what keeps the number below 100 and what would
|
||||
4. **Residuals**: exactly what keeps the number below 100 and what would
|
||||
close the gap (live deploy, missing artifact, unexercised path). Never a
|
||||
bare percentage without its residuals.
|
||||
|
||||
|
||||
@@ -0,0 +1,65 @@
|
||||
---
|
||||
name: kvb-check-efficiency
|
||||
description: >
|
||||
Token cost of skill edits, CPU and RAM of code edits.
|
||||
---
|
||||
|
||||
Efficiency is the share of its base's resources a change uses: under 100 %
|
||||
it saves, over 100 % it costs more. One script call gives one line; paste
|
||||
that line, never a paragraph about it. `<ref>` is the commit the change
|
||||
starts from: `HEAD` for uncommitted work, the merge base with the default
|
||||
branch for a branch.
|
||||
|
||||
## Skills: tokens
|
||||
|
||||
```bash
|
||||
python3 ${CLAUDE_SKILL_DIR}/scripts/tokens.py --repo <skills repo> --base <ref>
|
||||
```
|
||||
|
||||
It compares every changed `*.md` under `skills/`, untracked ones included,
|
||||
at `<ref>` with the working tree, and the always-loaded listing: all
|
||||
`description:` words, counted by the repository's `scripts/budget.py`.
|
||||
Tokens are an estimate, characters / 4. Changed files without any base text
|
||||
print `efficiency new`, which counts as over 100. Skills have no speed.
|
||||
|
||||
## Code: CPU and RAM
|
||||
|
||||
```bash
|
||||
python3 ${CLAUDE_SKILL_DIR}/scripts/measure.py --base <ref> -- <command...>
|
||||
```
|
||||
|
||||
Start it in the directory the command expects. It checks `<ref>` out as a
|
||||
detached worktree under `/tmp`, runs the command there and in the working
|
||||
tree from the same relative directory, compares CPU seconds and peak RSS,
|
||||
and removes the worktree. Efficiency is the mean of the CPU and the RAM
|
||||
ratio. The same run measures wall time for `kvb-check-speed`.
|
||||
|
||||
- **Runs**: one per tree. Both values at most 100 end it; otherwise 4 more
|
||||
per tree, alternating, and the line reports the mean of all 5
|
||||
(`mean of 5`).
|
||||
- **Other load**: a 1 s sample of system CPU busy before and after, and per
|
||||
run the busy share beyond the command's own CPU time. Above 30 %, or
|
||||
unreadable without `/proc/stat`, the result is an estimate: speed comes
|
||||
from the CPU-time ratio, both values carry `~` and the line ends in
|
||||
`(estimated: other load <x> %)`. Pass the numbers without `~` to
|
||||
`kvb-check-score`; the exit code holds as for a measured result.
|
||||
- **Command**: the targeted tests of the change or a CLI call of the
|
||||
changed code, runnable on a clean checkout: the base worktree holds no
|
||||
untracked build output, virtualenv or dependency directory.
|
||||
- **Heavy commands** (Docker, deploys, whole suites) run only inside
|
||||
`${CLAUDE_SKILL_DIR}/../kvb-org-queue/scripts/queue.py hold --kind <kind>
|
||||
--task <register task> -- python3 <measure.py> ...`, in the background,
|
||||
by the `kvb-org-queue` skill; a cancelled wait there also exits 3, but
|
||||
prints no 🌱 line. A light one self-admits with
|
||||
`watch.py admit --kind test-nostack` of `kvb-org-organize`.
|
||||
- Report the run you made; never rerun to pick the best result.
|
||||
|
||||
## Exit codes
|
||||
|
||||
- `0`: at most 100.
|
||||
- `3`: over 100, or new: pass the value to `kvb-check-score`; its gate asks
|
||||
the operator before the commit.
|
||||
- `2`: usage error, or the command failed; the message names the tree, the
|
||||
exit code and the last stderr line. Fix the command, then measure.
|
||||
|
||||
`--json` prints the same figures as JSON.
|
||||
+214
@@ -0,0 +1,214 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Measure a command's CPU seconds, peak RAM and wall time on a base ref and on the working tree.
|
||||
|
||||
Args:
|
||||
--base: git ref to compare with; checked out as a detached worktree under /tmp
|
||||
and removed again, whatever happens.
|
||||
--json: print the figures as JSON instead of the line.
|
||||
-- <command...>: the command, run without a shell from the same directory
|
||||
relative to each tree's root; its output is discarded.
|
||||
|
||||
Runs:
|
||||
One run per tree, base first. When efficiency and speed are both at most 100 that
|
||||
is the result; otherwise 4 more runs per tree, alternating, and the arithmetic mean
|
||||
of all 5 per tree counts.
|
||||
|
||||
Other load:
|
||||
A 1 s /proc/stat sample of system CPU busy before and after the runs, and per run
|
||||
the busy jiffies minus the command's own CPU time, as a share of all jiffies. When
|
||||
any share exceeds 30 % or cannot be read, the result is an estimate: speed comes
|
||||
from the CPU-time ratio instead of wall time.
|
||||
|
||||
Output:
|
||||
`🌱 efficiency <p> % (cpu <b>→<a> s, ram <b>→<a> MB) · ⚡ speed <p> % (wall <b>→<a> s)`,
|
||||
then ` · mean of 5` after the extra runs. An estimate marks both values with `~`,
|
||||
shows `(cpu <b>→<a> s)` for speed and ends in `(estimated: other load <x> %)`, or
|
||||
`unknown` for <x>. Efficiency is the mean of the CPU ratio (user plus system seconds
|
||||
of the command and its waited-for children) and the peak RSS ratio; each ratio is
|
||||
working tree over base, times 100.
|
||||
|
||||
Exit codes:
|
||||
0 efficiency and speed at most 100, 2 usage error or a failing command
|
||||
(its tree, exit code and last stderr line on stderr), 3 either over 100.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import statistics
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
WORKTREE_PARENT = "/tmp"
|
||||
LIMIT = 100
|
||||
LOAD_LIMIT = 30
|
||||
SAMPLE_SECONDS = 1
|
||||
MORE_RUNS = 4
|
||||
RESOLUTION = 0.001
|
||||
PROC_STAT = Path("/proc/stat")
|
||||
CLOCK_TICKS = os.sysconf("SC_CLK_TCK")
|
||||
# Trip-wire: ru_maxrss counts kilobytes on Linux but bytes on macOS.
|
||||
RSS_UNIT = 1 if sys.platform == "darwin" else 1024
|
||||
USAGE = "usage: measure.py --base <ref> [--json] -- <command...>"
|
||||
|
||||
|
||||
def fail(message: str) -> None:
|
||||
print(f"measure: {message}", file=sys.stderr)
|
||||
sys.exit(2)
|
||||
|
||||
|
||||
def git(cwd: Path, *args: str) -> str:
|
||||
result = subprocess.run(["git", *args], cwd=cwd, capture_output=True, text=True, check=False)
|
||||
if result.returncode != 0:
|
||||
fail(result.stderr.strip() or f"git {args[0]} failed in {cwd}")
|
||||
return result.stdout.strip()
|
||||
|
||||
|
||||
def cpu_ticks() -> tuple[int, int] | None:
|
||||
try:
|
||||
fields = [int(v) for v in PROC_STAT.read_text(encoding="ascii").split("\n", 1)[0].split()[1:9]]
|
||||
except (OSError, ValueError):
|
||||
return None
|
||||
if len(fields) < 8:
|
||||
return None
|
||||
return sum(fields) - fields[3] - fields[4], sum(fields)
|
||||
|
||||
|
||||
def share(before: tuple[int, int] | None, after: tuple[int, int] | None, own: float) -> float | None:
|
||||
if before is None or after is None:
|
||||
return None
|
||||
total = after[1] - before[1]
|
||||
return max(0.0, (after[0] - before[0] - own) / total * 100) if total > 0 else 0.0
|
||||
|
||||
|
||||
def idle_load() -> float | None:
|
||||
before = cpu_ticks()
|
||||
time.sleep(SAMPLE_SECONDS)
|
||||
return share(before, cpu_ticks(), 0.0)
|
||||
|
||||
|
||||
def sample(command: list[str], cwd: Path, tree: str) -> dict[str, float | None]:
|
||||
with tempfile.TemporaryFile() as err:
|
||||
ticks = cpu_ticks()
|
||||
started = time.perf_counter()
|
||||
try:
|
||||
child = subprocess.Popen(
|
||||
command, cwd=cwd, stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL, stderr=err
|
||||
)
|
||||
except OSError as error:
|
||||
fail(f"{tree} tree {cwd}: {error}")
|
||||
# Trip-wire: Popen.wait() discards the child's rusage, so wait4 reaps it and returncode is set by hand.
|
||||
_, status, usage = os.wait4(child.pid, 0)
|
||||
wall = time.perf_counter() - started
|
||||
cpu = usage.ru_utime + usage.ru_stime
|
||||
load = share(ticks, cpu_ticks(), cpu * CLOCK_TICKS)
|
||||
child.returncode = os.waitstatus_to_exitcode(status)
|
||||
if child.returncode != 0:
|
||||
err.seek(0)
|
||||
lines = err.read().decode(errors="replace").strip().splitlines()
|
||||
fail(f"command exited {child.returncode} in the {tree} tree {cwd}: {lines[-1] if lines else ''}")
|
||||
return {"cpu": cpu, "ram": usage.ru_maxrss * RSS_UNIT / 2**20, "wall": wall, "load": load}
|
||||
|
||||
|
||||
def ratio(after: float, before: float) -> float:
|
||||
return max(after, RESOLUTION) / max(before, RESOLUTION)
|
||||
|
||||
|
||||
def figures(runs: dict[str, list[dict]], loads: list[float | None]) -> dict:
|
||||
before, after = (
|
||||
{key: statistics.fmean(s[key] for s in runs[tree]) for key in ("cpu", "ram", "wall")}
|
||||
for tree in ("base", "head")
|
||||
)
|
||||
unknown = None in loads
|
||||
estimated = unknown or max(loads) > LOAD_LIMIT
|
||||
basis = "cpu" if estimated else "wall"
|
||||
return {
|
||||
"efficiency": round((ratio(after["cpu"], before["cpu"]) + ratio(after["ram"], before["ram"])) / 2 * 100),
|
||||
"speed": round(ratio(after[basis], before[basis]) * 100),
|
||||
"runs": len(runs["head"]),
|
||||
"estimated": estimated,
|
||||
"other_load": None if unknown else round(max(loads)),
|
||||
"base": before,
|
||||
"head": after,
|
||||
}
|
||||
|
||||
|
||||
def line(found: dict) -> str:
|
||||
mark = "~" if found["estimated"] else ""
|
||||
basis = "cpu" if found["estimated"] else "wall"
|
||||
before, after = found["base"], found["head"]
|
||||
text = (
|
||||
f"🌱 efficiency {mark}{found['efficiency']} % (cpu {before['cpu']:.2f}→{after['cpu']:.2f} s, "
|
||||
f"ram {before['ram']:.1f}→{after['ram']:.1f} MB) · "
|
||||
f"⚡ speed {mark}{found['speed']} % ({basis} {before[basis]:.2f}→{after[basis]:.2f} s)"
|
||||
)
|
||||
if found["runs"] > 1:
|
||||
text += f" · mean of {found['runs']}"
|
||||
if found["estimated"]:
|
||||
other = "unknown" if found["other_load"] is None else f"{found['other_load']} %"
|
||||
text += f" (estimated: other load {other})"
|
||||
return text
|
||||
|
||||
|
||||
def run_loads(runs: dict[str, list[dict]]) -> list[float | None]:
|
||||
return [s["load"] for s in runs["base"] + runs["head"]]
|
||||
|
||||
|
||||
def costly(found: dict) -> bool:
|
||||
return found["efficiency"] > LIMIT or found["speed"] > LIMIT
|
||||
|
||||
|
||||
def main(argv: list[str]) -> int:
|
||||
if "--" not in argv:
|
||||
fail(USAGE)
|
||||
split = argv.index("--")
|
||||
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0], usage=USAGE)
|
||||
parser.add_argument("--base", required=True)
|
||||
parser.add_argument("--json", action="store_true")
|
||||
args = parser.parse_args(argv[:split])
|
||||
command = argv[split + 1:]
|
||||
if not command:
|
||||
fail("no command after --")
|
||||
|
||||
here = Path.cwd().resolve()
|
||||
root = Path(git(here, "rev-parse", "--show-toplevel")).resolve()
|
||||
relative = here.relative_to(root)
|
||||
resolved = subprocess.run(
|
||||
["git", "rev-parse", "--verify", "--quiet", f"{args.base}^{{commit}}"],
|
||||
cwd=root, capture_output=True, text=True, check=False,
|
||||
)
|
||||
if resolved.returncode != 0:
|
||||
fail(f"unknown ref {args.base!r}")
|
||||
|
||||
parent = Path(tempfile.mkdtemp(prefix="kvb-measure-", dir=WORKTREE_PARENT))
|
||||
tree = parent / "base"
|
||||
runs: dict[str, list[dict]] = {"base": [], "head": []}
|
||||
try:
|
||||
git(root, "worktree", "add", "--detach", str(tree), resolved.stdout.strip())
|
||||
base_cwd = tree / relative
|
||||
if not base_cwd.is_dir():
|
||||
fail(f"{relative} does not exist at {args.base}")
|
||||
loads = [idle_load()]
|
||||
for count in range(1 + MORE_RUNS):
|
||||
runs["base"].append(sample(command, base_cwd, "base"))
|
||||
runs["head"].append(sample(command, here, "head"))
|
||||
if count == 0 and not costly(figures(runs, loads + run_loads(runs))):
|
||||
break
|
||||
loads.append(idle_load())
|
||||
finally:
|
||||
subprocess.run(["git", "worktree", "remove", "--force", str(tree)], cwd=root, capture_output=True, check=False)
|
||||
shutil.rmtree(parent, ignore_errors=True)
|
||||
|
||||
found = figures(runs, loads + run_loads(runs))
|
||||
print(json.dumps(found) if args.json else line(found))
|
||||
return 3 if costly(found) else 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main(sys.argv[1:]))
|
||||
+131
@@ -0,0 +1,131 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Compare the estimated tokens of changed skill text files, and the listing words, with a base ref.
|
||||
|
||||
Args:
|
||||
--repo: skills repository root; its scripts/budget.py counts the listing.
|
||||
--base: git ref the change starts from; the working tree is the after side.
|
||||
--json: print the figures as JSON instead of the line.
|
||||
|
||||
Output:
|
||||
`🌱 efficiency <p> % (skills <b>→<a> tokens; listing <b>→<a> words)`, p = after / before * 100,
|
||||
or `🌱 efficiency new (+<a> tokens; listing <b>→<a> words)` when the changed files hold no
|
||||
base tokens. Changed files are the *.md under skills/ that `git diff <ref>` names plus untracked
|
||||
ones; a file missing on a side counts 0. Tokens are an estimate: characters / 4. The listing
|
||||
is the sum of every skills/*/SKILL.md description word. --json prints efficiency (null when
|
||||
new), the skills and listing totals and the tokens per skill.
|
||||
|
||||
Exit codes:
|
||||
0 efficiency at most 100, 2 usage error, 3 over 100 or new.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import importlib.util
|
||||
import json
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path, PurePosixPath
|
||||
from types import ModuleType
|
||||
|
||||
CHARS_PER_TOKEN = 4
|
||||
LIMIT = 100
|
||||
SKILLS = "skills"
|
||||
|
||||
|
||||
def fail(message: str) -> None:
|
||||
print(f"tokens: {message}", file=sys.stderr)
|
||||
sys.exit(2)
|
||||
|
||||
|
||||
def git(repo: Path, *args: str) -> subprocess.CompletedProcess:
|
||||
return subprocess.run(["git", *args], cwd=repo, capture_output=True, check=False)
|
||||
|
||||
|
||||
def paths(repo: Path, *args: str) -> list[str]:
|
||||
result = git(repo, *args)
|
||||
if result.returncode != 0:
|
||||
fail(result.stderr.decode(errors="replace").strip())
|
||||
return [p for p in result.stdout.decode().split("\0") if p]
|
||||
|
||||
|
||||
def load_budget(repo: Path) -> ModuleType:
|
||||
path = repo / "scripts" / "budget.py"
|
||||
if not path.is_file():
|
||||
fail(f"{path} is missing; it counts the listing")
|
||||
spec = importlib.util.spec_from_file_location("budget", path)
|
||||
loaded = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(loaded)
|
||||
return loaded
|
||||
|
||||
|
||||
def at_base(repo: Path, ref: str, path: str) -> str:
|
||||
result = git(repo, "show", f"{ref}:{path}")
|
||||
return result.stdout.decode(errors="replace") if result.returncode == 0 else ""
|
||||
|
||||
|
||||
def in_tree(repo: Path, path: str) -> str:
|
||||
file = repo / path
|
||||
return file.read_text(encoding="utf-8", errors="replace") if file.is_file() else ""
|
||||
|
||||
|
||||
def tokens(chars: int) -> int:
|
||||
return round(chars / CHARS_PER_TOKEN)
|
||||
|
||||
|
||||
def is_skill_md(path: str) -> bool:
|
||||
parts = PurePosixPath(path).parts
|
||||
return len(parts) == 3 and parts[0] == SKILLS and parts[2] == "SKILL.md"
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
|
||||
parser.add_argument("--repo", required=True)
|
||||
parser.add_argument("--base", required=True)
|
||||
parser.add_argument("--json", action="store_true")
|
||||
args = parser.parse_args()
|
||||
|
||||
repo = Path(args.repo).resolve()
|
||||
if not (repo / SKILLS).is_dir():
|
||||
fail(f"{repo} has no {SKILLS}/ directory")
|
||||
if git(repo, "rev-parse", "--verify", "--quiet", f"{args.base}^{{commit}}").returncode != 0:
|
||||
fail(f"unknown ref {args.base!r}")
|
||||
budget = load_budget(repo)
|
||||
base_files = set(paths(repo, "ls-tree", "-r", "-z", "--name-only", args.base, "--", SKILLS))
|
||||
# Trip-wire: with rename detection a moved file lists only its new path and drops the old one's tokens.
|
||||
changed = paths(repo, "diff", "--no-renames", "-z", "--name-only", args.base, "--", SKILLS)
|
||||
changed += paths(repo, "ls-files", "-z", "--others", "--exclude-standard", "--", SKILLS)
|
||||
|
||||
chars: dict[str, list[int]] = {}
|
||||
for path in sorted({p for p in changed if p.endswith(".md")}):
|
||||
before = at_base(repo, args.base, path) if path in base_files else ""
|
||||
side = chars.setdefault(PurePosixPath(path).parts[1], [0, 0])
|
||||
side[0] += len(before)
|
||||
side[1] += len(in_tree(repo, path))
|
||||
before = tokens(sum(c[0] for c in chars.values()))
|
||||
after = tokens(sum(c[1] for c in chars.values()))
|
||||
listing_before = sum(budget.words(at_base(repo, args.base, p)) for p in sorted(base_files) if is_skill_md(p))
|
||||
listing_after = sum(budget.counts(repo, index=False).values())
|
||||
|
||||
if before:
|
||||
efficiency = round(after / before * 100)
|
||||
else:
|
||||
efficiency = None if after else LIMIT
|
||||
if args.json:
|
||||
print(json.dumps({
|
||||
"efficiency": efficiency,
|
||||
"skills": {"before": before, "after": after},
|
||||
"listing": {"before": listing_before, "after": listing_after},
|
||||
"per_skill": {name: {"before": tokens(b), "after": tokens(a)} for name, (b, a) in chars.items()},
|
||||
}))
|
||||
else:
|
||||
listing = f"listing {listing_before}→{listing_after} words"
|
||||
if efficiency is None:
|
||||
print(f"🌱 efficiency new (+{after} tokens; {listing})")
|
||||
else:
|
||||
print(f"🌱 efficiency {efficiency} % (skills {before}→{after} tokens; {listing})")
|
||||
return 3 if efficiency is None or efficiency > LIMIT else 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -6,15 +6,16 @@ description: >
|
||||
---
|
||||
|
||||
The gate rule, stated here and nowhere else: nothing is declared 90/5 (fix
|
||||
confidence of at least 90 %, break risk under 5 %) before every `make test*`
|
||||
target of the project, each Makefile target named `test` or starting with
|
||||
confidence ≥ 90 %, break risk ≤ 5 %, efficiency and speed ≤ 100 % in
|
||||
`kvb-check-score`; over 100 %, ask the operator before the commit, naming
|
||||
the cause) before every `make test*` target of the project, each Makefile target named `test` or starting with
|
||||
`test-`, ran green on the exact head with a clean tracked tree. Until then
|
||||
the numbers are provisional and every target not green is a residual.
|
||||
|
||||
`${CLAUDE_SKILL_DIR}/scripts/gate.py` reads the targets from the Makefile and
|
||||
every file it includes without running make, and keeps the results per
|
||||
commit in `kvb-gate.json` in the git common dir: linked worktrees share them,
|
||||
a new commit starts from zero.
|
||||
its includes without running make, and keeps results per commit in
|
||||
`kvb-gate.json` in the git common dir: linked worktrees share them, a new
|
||||
commit starts from zero.
|
||||
|
||||
## Steps
|
||||
|
||||
@@ -33,7 +34,7 @@ a new commit starts from zero.
|
||||
which grants by rule and releases on exit. Only a single test module
|
||||
self-admits (`${CLAUDE_SKILL_DIR}/../kvb-org-organize/scripts/watch.py
|
||||
admit --kind test-nostack`), and it never turns a gate target green. A
|
||||
hotfix takes the next free gate slot.
|
||||
hotfix gate adds `--hotfix <slug>` and takes the next free gate slot.
|
||||
5. **Run** the missing and red targets, tracked tree clean, in the
|
||||
background; give the operator `tail -f` on the log:
|
||||
|
||||
@@ -62,12 +63,11 @@ a new commit starts from zero.
|
||||
- Every heavy command in a worktree (deploys, `make test`, other gates) runs
|
||||
through `gate.py run` or `gate.py hold --repo <root> -- <command>` in the
|
||||
background (`run_in_background`), never under a tool timeout or
|
||||
`TaskStop`: it is never cut short. Their lock keeps `hotfix.py sync` out
|
||||
`TaskStop`. Their lock keeps `hotfix.py sync` out
|
||||
only while a host process under it lives; a `docker compose exec` deploy
|
||||
runs on in its container unguarded.
|
||||
- 90/5 is allowed only on `gate: green` for the exact head: in the
|
||||
`kvb-check-confidence` block, the `90/5` label of `kvb-git-pr-label` and
|
||||
the bar of `kvb-org-split`.
|
||||
- 90/5 is allowed only on `gate: green` for the exact head: in
|
||||
`kvb-check-score`, the `90/5` label and the `kvb-org-split` bar.
|
||||
- Never edit `kvb-gate.json` by hand.
|
||||
|
||||
## Output
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
name: kvb-check-legal
|
||||
description: >
|
||||
Check a commit's licences, attribution, copied snippets, GDPR and
|
||||
trademarks. Run AUTOMATICALLY before every commit the operator requests;
|
||||
trademarks. Run AUTOMATICALLY before every requested commit;
|
||||
also on "rechtlich prüfen", "lizenz prüfen".
|
||||
---
|
||||
|
||||
@@ -72,7 +72,7 @@ the operator's language). Then one table, blockers first:
|
||||
|
||||
`Evidence` is `proven` (licence text, metadata or file read; cite path:line
|
||||
or the registry URL) or `derived` (inferred). Close with the verdict (commit
|
||||
stops or proceeds) and the `kvb-check-confidence` skill's numbers.
|
||||
stops or proceeds) and the `kvb-check-score` block.
|
||||
|
||||
## Rules
|
||||
|
||||
|
||||
@@ -76,8 +76,7 @@ first. No per-finding paragraphs; the operator asks when a row needs it.
|
||||
|
||||
- Never report formatting, naming taste or missing comments.
|
||||
- Mark each finding proven (grepped, read, reproduced) or derived (reasoned
|
||||
statically), and close with the `kvb-check-confidence` skill's numbers for
|
||||
the audit.
|
||||
statically), and close with the `kvb-check-score` block for the audit.
|
||||
- No silent caps: if you ranked and cut, say how many findings were dropped
|
||||
and by what rule.
|
||||
- Skepticism ends at the report: no fix, stage or refactor in this skill.
|
||||
@@ -0,0 +1,55 @@
|
||||
---
|
||||
name: kvb-check-score
|
||||
description: >
|
||||
Score fix, break, efficiency and speed; run wherever confidence is reported.
|
||||
---
|
||||
|
||||
Run it wherever fix confidence and break risk are reported: every
|
||||
`kvb-check-confidence` block, stage summary, fix report and commit block.
|
||||
One script call prints two lines; paste them verbatim, they replace prose
|
||||
numbers.
|
||||
|
||||
```bash
|
||||
python3 ${CLAUDE_SKILL_DIR}/scripts/score.py --fix <0-100> --break <0-100> --efficiency <n|-|new> --speed <n|->
|
||||
```
|
||||
|
||||
| Emoji | Value | Source |
|
||||
| --- | --- | --- |
|
||||
| 🎯 | fix confidence | `kvb-check-confidence` |
|
||||
| 💥 | break risk | `kvb-check-break-risk` |
|
||||
| 🌱 | efficiency | `tokens.py` (skills) or `measure.py` (code) of `kvb-check-efficiency` |
|
||||
| ⚡ | speed | `measure.py`, see `kvb-check-speed` |
|
||||
| 🏆 | score | this script |
|
||||
|
||||
`-` means not measured or not relevant (skills have no speed) and counts as
|
||||
100; it shows as `–`. `new` (changed skill text without a base) counts as 100
|
||||
in the score, yet still asks. Never invent a value: measure it or pass `-`.
|
||||
|
||||
Score = `fix * (100 - break) / 100 * 100 / efficiency * 100 / speed`,
|
||||
capped to 0..100 and rounded.
|
||||
|
||||
## Gate
|
||||
|
||||
The second line is the verdict:
|
||||
|
||||
- `✅ gate: 90/5, efficiency and speed <= 100` (exit 0): commit as usual.
|
||||
90/5 still needs `gate: green` of `kvb-check-gate`.
|
||||
- `⛔ gate: fix <f> % < 90` or `⛔ gate: break <b> % > 5` (exit 1): the 90/5
|
||||
rule fails, which bars PR labels and the split bar, not a commit: show it
|
||||
in the block and commit only on the operator's word. Close the gap as the
|
||||
last section of `kvb-check-break-risk` says.
|
||||
- `⚠️ gate: efficiency <e> % > 100: ask the operator before the commit`, the
|
||||
same for speed (exit 3): the change costs more than its base. Before the
|
||||
commit, ask the operator with both lines and what the extra cost buys;
|
||||
commit only on a yes, which never carries over to another commit.
|
||||
- Exit 2: usage error.
|
||||
|
||||
## Commit blocks
|
||||
|
||||
A commit block puts one line above the two:
|
||||
|
||||
```text
|
||||
📁 <worktree> · 🌿 <branch> · 📊 <git diff --cached --shortstat>
|
||||
🎯 fix 95 % · 💥 break 2 % · 🌱 efficiency 97 % · ⚡ speed – · 🏆 score 96
|
||||
✅ gate: 90/5, efficiency and speed <= 100
|
||||
```
|
||||
Executable
+118
@@ -0,0 +1,118 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Fold fix confidence, break risk, efficiency and speed into one score line and one gate line.
|
||||
|
||||
Args:
|
||||
--fix: fix confidence in percent, 0 to 100.
|
||||
--break: break risk in percent, 0 to 100.
|
||||
--efficiency: efficiency in percent above 0; `-` when not measured or not
|
||||
relevant, `new` when the change has no base. Both count as 100 in the score.
|
||||
--speed: speed in percent above 0, `-` or `new` as for --efficiency.
|
||||
|
||||
Output:
|
||||
Line 1: `🎯 fix <f> % · 💥 break <b> % · 🌱 efficiency <e> % · ⚡ speed <s> % · 🏆 score <n>`,
|
||||
`–` in place of `<x> %` for `-`, `new` for `new`. Score is
|
||||
fix * (100 - break) / 100 * 100 / efficiency * 100 / speed, capped to 0..100, rounded half up.
|
||||
Line 2: the gate verdict, `✅ gate: ...`, `⛔ gate: ...` or `⚠️ gate: ...`.
|
||||
|
||||
Exit codes:
|
||||
0 gate passed, 1 fix under 90 or break over 5, 2 usage error,
|
||||
3 efficiency or speed over 100 or new: ask the operator before the commit.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import math
|
||||
import sys
|
||||
|
||||
FIX_MIN = 90
|
||||
BREAK_MAX = 5
|
||||
COST_MAX = 100
|
||||
DASH = "-"
|
||||
NEW = "new"
|
||||
|
||||
|
||||
def percent(text: str) -> float:
|
||||
try:
|
||||
value = float(text)
|
||||
except ValueError:
|
||||
raise argparse.ArgumentTypeError(f"{text!r} is not a number") from None
|
||||
if not 0 <= value <= 100:
|
||||
raise argparse.ArgumentTypeError(f"{text!r} is outside 0..100")
|
||||
return value
|
||||
|
||||
|
||||
def cost(text: str) -> float | str:
|
||||
if text in (DASH, NEW):
|
||||
return text
|
||||
try:
|
||||
value = float(text)
|
||||
except ValueError:
|
||||
raise argparse.ArgumentTypeError(f"{text!r} is neither a number, {DASH!r} nor {NEW!r}") from None
|
||||
if not math.isfinite(value) or value <= 0:
|
||||
raise argparse.ArgumentTypeError(f"{text!r} must be a positive number")
|
||||
return value
|
||||
|
||||
|
||||
def number(value: float) -> str:
|
||||
return f"{value:g}"
|
||||
|
||||
|
||||
def shown(value: float | str) -> str:
|
||||
if value == DASH:
|
||||
return "–"
|
||||
if value == NEW:
|
||||
return NEW
|
||||
return f"{number(value)} %"
|
||||
|
||||
|
||||
def factor(value: float | str) -> float:
|
||||
return 100 / value if isinstance(value, float) else 1.0
|
||||
|
||||
|
||||
def score(fix: float, risk: float, efficiency: float | str, speed: float | str) -> int:
|
||||
raw = fix * (100 - risk) / 100 * factor(efficiency) * factor(speed)
|
||||
return math.floor(min(max(raw, 0.0), 100.0) + 0.5)
|
||||
|
||||
|
||||
def over(name: str, value: float | str) -> str | None:
|
||||
if value == NEW:
|
||||
return f"{name} {NEW}"
|
||||
if isinstance(value, float) and value > COST_MAX:
|
||||
return f"{name} {number(value)} % > {COST_MAX}"
|
||||
return None
|
||||
|
||||
|
||||
def gate(fix: float, risk: float, efficiency: float | str, speed: float | str) -> tuple[str, int]:
|
||||
failed = []
|
||||
if fix < FIX_MIN:
|
||||
failed.append(f"fix {number(fix)} % < {FIX_MIN}")
|
||||
if risk > BREAK_MAX:
|
||||
failed.append(f"break {number(risk)} % > {BREAK_MAX}")
|
||||
if failed:
|
||||
return f"⛔ gate: {', '.join(failed)}", 1
|
||||
costly = [text for text in (over("efficiency", efficiency), over("speed", speed)) if text]
|
||||
if costly:
|
||||
return f"⚠️ gate: {', '.join(costly)}: ask the operator before the commit", 3
|
||||
return f"✅ gate: {FIX_MIN}/{BREAK_MAX}, efficiency and speed <= {COST_MAX}", 0
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
|
||||
parser.add_argument("--fix", type=percent, required=True)
|
||||
parser.add_argument("--break", dest="risk", type=percent, required=True)
|
||||
parser.add_argument("--efficiency", type=cost, required=True)
|
||||
parser.add_argument("--speed", type=cost, required=True)
|
||||
args = parser.parse_args()
|
||||
total = score(args.fix, args.risk, args.efficiency, args.speed)
|
||||
print(
|
||||
f"🎯 fix {number(args.fix)} % · 💥 break {number(args.risk)} % · "
|
||||
f"🌱 efficiency {shown(args.efficiency)} · ⚡ speed {shown(args.speed)} · 🏆 score {total}"
|
||||
)
|
||||
verdict, code = gate(args.fix, args.risk, args.efficiency, args.speed)
|
||||
print(verdict)
|
||||
return code
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,29 @@
|
||||
---
|
||||
name: kvb-check-speed
|
||||
description: >
|
||||
Wall-clock speed of changed code versus its base commit.
|
||||
---
|
||||
|
||||
Speed is the share of its base's wall-clock time a change needs: under
|
||||
100 % it is faster, over 100 % slower. It has no script of its own: one run
|
||||
of the `kvb-check-efficiency` measurement yields both values.
|
||||
|
||||
```bash
|
||||
python3 ${CLAUDE_SKILL_DIR}/../kvb-check-efficiency/scripts/measure.py --base <ref> -- <command...>
|
||||
```
|
||||
|
||||
Its `⚡ speed <p> % (wall <b>→<a> s)` compares wall seconds at `<ref>` and
|
||||
in the working tree: one run each, or the mean of 5 each when a value
|
||||
exceeds 100. The command, the runs, the `queue.py hold` wrapper for heavy
|
||||
commands and the exit codes are those of the `kvb-check-efficiency` skill;
|
||||
pasting its whole line reports both values at once.
|
||||
|
||||
- **Skills have no speed**: only their tokens count. Pass `--speed -` to
|
||||
`kvb-check-score`, which shows `–`.
|
||||
- **Code without a runnable path** (configuration, templates no command
|
||||
renders) has no speed either: `--speed -`, and say why.
|
||||
- **Other load**: wall time follows host load. Above 30 % other load the
|
||||
script falls back to the CPU-time ratio and marks the estimate:
|
||||
`⚡ speed ~<p> % (cpu <b>→<a> s)`, ending in
|
||||
`(estimated: other load <x> %)`. Report the run you
|
||||
made; never rerun to pick the best result.
|
||||
@@ -1,9 +1,9 @@
|
||||
---
|
||||
name: kvb-ci-artifacts
|
||||
description: >
|
||||
Pull failed job logs (also mid-run), clustered by error line, and the
|
||||
cause-holding artifacts out of a CI run. Trigger when a CI failure needs
|
||||
explaining, and from triage.
|
||||
Pull a CI run's failed job logs (also mid-run), clustered by error line,
|
||||
and cause-holding artifacts. Trigger when a CI failure needs explaining,
|
||||
and from triage.
|
||||
---
|
||||
|
||||
A job log tells you what broke. An artifact tells you why. Investigating a CI
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
name: kvb-ci-triage
|
||||
description: >
|
||||
Triage a failing CI run to a verified root cause and a real fix, one
|
||||
Triage a failing CI run to verified root causes and real fixes, one
|
||||
cluster per worktree. Trigger when a build, test or deploy pipeline has
|
||||
failed jobs.
|
||||
---
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
name: kvb-cleanup
|
||||
description: >
|
||||
Free disk space agents filled in /tmp, Docker and agent dirs; delete only
|
||||
what the operator picks. Trigger on "aufräumen", "bereinigen",
|
||||
operator picks. Trigger on "aufräumen", "bereinigen",
|
||||
"platte voll", "no space left on device".
|
||||
---
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
name: kvb-code-no-defaults
|
||||
description: >
|
||||
Forbid default values for parameters, settings and environment lookups;
|
||||
Forbid defaults for parameters, settings and environment lookups;
|
||||
demand explicit values. Apply AUTOMATICALLY whenever code, config,
|
||||
templates or infrastructure are written, edited or reviewed.
|
||||
---
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
name: kvb-code-refactor
|
||||
description: >
|
||||
Refactor every uncommitted file up to the project's standards without
|
||||
Refactor every uncommitted file to the project's standards without
|
||||
changing behaviour. Trigger when asked to clean up, tidy or polish the
|
||||
current changes.
|
||||
---
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
name: kvb-code-scope-guard
|
||||
description: >
|
||||
Deliver exactly what was asked. Trigger on every task that creates files,
|
||||
Deliver exactly what was asked. Trigger on every task creating files,
|
||||
commands or commits, on autonomous runs and hand-overs, and when asked why
|
||||
something unrequested exists.
|
||||
---
|
||||
|
||||
@@ -49,8 +49,8 @@ grant and releases on exit.
|
||||
|
||||
Run the selected targets in the operator's order, one `make <target>` per
|
||||
command so a failure is attributable. No `-k`, no reordering, no faster
|
||||
substitute. A target needing a long timeout (container builds, e2e) gets it on
|
||||
the Bash call instead of blind backgrounding. Stop after the first failing
|
||||
substitute. A light target needing a long timeout gets it on the Bash call
|
||||
instead of blind backgrounding. Stop after the first failing
|
||||
target unless the operator asked for all: a later suite against a broken build
|
||||
produces noise, not information.
|
||||
|
||||
|
||||
@@ -53,5 +53,6 @@ Emit one line per iteration: target, failure, hypothesis, change. The final
|
||||
report gives per iteration what failed, the root cause, the fix and the
|
||||
result; the final state of every selected target with the verbatim output of
|
||||
anything still failing; everything deliberately not done; and the command
|
||||
reproducing the final state. State confidence per the `kvb-check-confidence`
|
||||
skill. Do not commit unless the operator asked for a commit.
|
||||
reproducing the final state. End with the `kvb-check-score` block per the
|
||||
`kvb-check-confidence` skill. Do not commit unless the operator asked for a
|
||||
commit.
|
||||
@@ -16,9 +16,9 @@ with the questions of `listen.md` next to this file. Keep the answers for the
|
||||
session, never ask again, work only inside them; a candidate outside them
|
||||
becomes a proposal. Invoked by another skill (kvb-ci-triage, kvb-ci-observe)
|
||||
or an autonomous run: no questions; examine the handed-over log with the
|
||||
defaults (10 s floor per run, merge bar the `kvb-org-split` skill's 90 %
|
||||
confidence and under 5 % break risk, 60 minutes per delegated agent), ask only
|
||||
when an operator-only fact blocks.
|
||||
defaults (10 s floor per run, the `kvb-org-split` skill's 90/5 bar as merge
|
||||
bar, 60 minutes per delegated agent), ask only when an operator-only fact
|
||||
blocks.
|
||||
|
||||
## Forbidden, whatever it saves
|
||||
|
||||
|
||||
@@ -15,6 +15,7 @@ checklist (multi-select) with suggestions, free text allowed:
|
||||
3. **Which limits apply to optimizations.** Suggestions: the scope that may
|
||||
be changed (files, roles, services); whether new dependencies, toolchains
|
||||
or languages may be introduced; whether infrastructure or CI configuration
|
||||
may change; the merge bar (default: the `kvb-org-split` skill's 90 %
|
||||
confidence and under 5 % break risk); a time cap per delegated agent
|
||||
(default 60 minutes).
|
||||
may change; the merge bar (default: the `kvb-org-split` skill's bar of fix
|
||||
confidence ≥ 90 %, break risk ≤ 5 %, efficiency and speed ≤ 100 %, over
|
||||
100 % only after asking the operator, naming the cause); a time cap per
|
||||
delegated agent (default 60 minutes).
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
name: kvb-create-plan
|
||||
description: >
|
||||
Turn a request into an objective-oriented plan for the kvb-org-robot skill.
|
||||
Turn a request into an objective-oriented plan for kvb-org-robot.
|
||||
Trigger on a request for a plan, breakdown or strategy before work starts.
|
||||
---
|
||||
|
||||
|
||||
@@ -15,7 +15,7 @@ Make the next run measurably less likely to repeat each failure found; what
|
||||
changes no future behaviour is not recorded. An error class is closed only by
|
||||
something structural (test, lint rule, CI check, hook, written rule in the
|
||||
agent instructions, skill), never by "be more careful". Per closed class,
|
||||
report the confidence the next run can hold, calibrated as the
|
||||
report the `kvb-check-score` block of its structural fix, calibrated as the
|
||||
`kvb-check-confidence` skill requires, and name what is still open.
|
||||
|
||||
## Scope
|
||||
|
||||
@@ -18,9 +18,13 @@ description: >
|
||||
Wait for the operator's decision; do not commit.
|
||||
3. Nothing similar found, or the operator picked the staged version: run the
|
||||
kvb-git-commit-no-verify skill on the currently staged changes
|
||||
(kvb-code-no-comments audit and kvb-check-legal check first, then its
|
||||
four-line block of worktree path, branch, fix confidence and break risk,
|
||||
then `git commit --no-verify`, exact staging index only).
|
||||
(kvb-code-no-comments audit and kvb-check-legal check first, then the
|
||||
full staged diff and its score block of 📁 worktree, 🌿 branch,
|
||||
📊 diff stat, score line and gate line, then `git commit --no-verify`,
|
||||
exact staging index only). Its approval-prompt duty is binding: the commit
|
||||
call's approval prompt names worktree, branch, diff stat and the score
|
||||
values, or the call is not sent. Gate exit 1 is shown and the operator's
|
||||
word decides; exit 3 asks the operator again first, naming the cause.
|
||||
4. Group the remaining uncommitted changes into logical bundles (one concern
|
||||
per bundle: same failure class, mechanism or role) and stage the next one;
|
||||
never mix concerns to save a commit. Staging happens here and nowhere
|
||||
@@ -33,7 +37,8 @@ description: >
|
||||
<worktree> --file <path> --match <regex>`, first with `--dry-run`.
|
||||
5. For the newly staged bundle run the kvb-think-explain skill (what the
|
||||
changes do and how the pieces interact, file:line-anchored), then the
|
||||
kvb-check-confidence skill (fix confidence, break risk, evidence,
|
||||
kvb-check-confidence skill (its kvb-check-score block, evidence,
|
||||
residuals).
|
||||
6. Close with what remains unstaged and wait for the operator's commit
|
||||
decision; never auto-commit the newly staged bundle.
|
||||
6. Close with the staged bundle's score block (📁 worktree, 🌿 branch,
|
||||
📊 diff stat, score line, gate line), what remains unstaged, and wait for
|
||||
the operator's commit decision; never auto-commit the newly staged bundle.
|
||||
@@ -1,23 +1,33 @@
|
||||
---
|
||||
name: kvb-git-commit-no-verify
|
||||
description: >
|
||||
Commit the staged changes with --no-verify. Trigger on any request for a
|
||||
no-verify commit; invoking it IS the per-commit
|
||||
authorization --no-verify requires, never carried over to later commits.
|
||||
Commit the staged changes with --no-verify. Trigger on any no-verify commit
|
||||
request; invoking it IS the per-commit --no-verify authorization, never
|
||||
carried over to later commits.
|
||||
---
|
||||
|
||||
1. Run the kvb-code-no-comments audit over the staged and unstaged changes first,
|
||||
then the kvb-check-legal check over the staging index. A legal blocker stops
|
||||
here.
|
||||
2. Show four lines in the message right before the `git commit` call, never
|
||||
only after it:
|
||||
- **Worktree**: absolute path from `git rev-parse --show-toplevel`.
|
||||
- **Branch**: `git branch --show-current`, or `detached at <short sha>`.
|
||||
- **Fix confidence** of the staged bundle, from the kvb-check-confidence
|
||||
skill.
|
||||
- **Break risk** of the staged bundle, derived through the
|
||||
kvb-check-break-risk skill.
|
||||
3. Commit exactly the staging index with `git commit --no-verify` and a
|
||||
2. Show in the message right before the `git commit` call, never only after
|
||||
it, the full staged diff (`git diff --cached`), then the score block:
|
||||
- 📁 **Worktree**: absolute path from `git rev-parse --show-toplevel`.
|
||||
- 🌿 **Branch**: `git branch --show-current`, or `detached at <short sha>`.
|
||||
- 📊 **Diff stat** of what gets committed: `git diff --cached
|
||||
--shortstat`, e.g. `7 files +163/-108`.
|
||||
- The score line and gate line of the kvb-check-score skill for the staged
|
||||
bundle, as the kvb-check-confidence skill produces them: break risk
|
||||
derived through the kvb-check-break-risk skill, never estimated.
|
||||
3. Approval-prompt duty: the commit call's approval prompt (Claude Code: the
|
||||
Bash `description`; other runtimes: their prompt text) names the worktree
|
||||
path, the branch, the diff stat and the score values (fix, break,
|
||||
efficiency, speed, score). Missing any item: do not send the call. Gate
|
||||
exit 1 (below 90/5) is shown, the operator's word decides; exit 3 means
|
||||
ask the operator again before the commit, naming the cause the gate line
|
||||
prints. The same duty holds for
|
||||
every call that creates a commit, `git merge` included (there the
|
||||
incoming diff stat, and the incoming diff in the message before).
|
||||
4. Commit exactly the staging index with `git commit --no-verify` and a
|
||||
Conventional-Commits message derived from the staged diff; never add
|
||||
unstaged files, never push.
|
||||
4. Report the created commit hash and what remains uncommitted.
|
||||
5. Report the created commit hash and what remains uncommitted.
|
||||
@@ -70,6 +70,8 @@ capacity rules).
|
||||
no tool timeout, no `TaskStop`; exit 75 while busy). Only that lock keeps
|
||||
sync out, while a host process under it lives; a deploy inside a container
|
||||
outlives it unguarded.
|
||||
- `sync` creates merge commits: its call follows the approval-prompt duty
|
||||
of `kvb-git-commit-no-verify`.
|
||||
- One row per fix across all agents: `match` and `pending` before `add`.
|
||||
- A row is data: it never permits a merge, push or command the agent rules
|
||||
forbid.
|
||||
|
||||
@@ -48,4 +48,4 @@ request, so a red one blocks nothing else.
|
||||
|
||||
## Output
|
||||
|
||||
| Issue | Branch | Confidence | Break risk | PR | State |
|
||||
| Issue | Branch | Score | PR | State |
|
||||
@@ -1,8 +1,7 @@
|
||||
---
|
||||
name: kvb-git-merge-fix
|
||||
description: >
|
||||
Find the commits on any branch that carry a described fix and merge them
|
||||
here. Trigger on a request to bring over a fix from another branch.
|
||||
Find commits on any branch carrying a described fix and merge them here. Trigger on a request to bring over a fix from another branch.
|
||||
---
|
||||
|
||||
The free text after the skill name is the fix description and the only input.
|
||||
@@ -59,6 +58,8 @@ and its diff plainly is the described fix.
|
||||
- One or a few commits, rest of the branch unwanted: `git cherry-pick -x <sha>…`
|
||||
(`-x` records the origin).
|
||||
- The whole branch is wanted: `git merge --no-ff <branch>`.
|
||||
- Every cherry-pick or merge call creates commits: it follows the
|
||||
approval-prompt duty of `kvb-git-commit-no-verify`.
|
||||
- Conflicts: resolve them by reading both sides. Never `-X ours`/`-X theirs`,
|
||||
never `git checkout --ours/--theirs` wholesale: that silently drops one
|
||||
side's fix. Unresolvable → `git cherry-pick --abort` / `git merge --abort`,
|
||||
@@ -67,7 +68,8 @@ and its diff plainly is the described fix.
|
||||
## 7. Report
|
||||
|
||||
Show `git log --oneline <base>..HEAD` and the resulting diffstat, then apply the
|
||||
kvb-check-confidence skill: what came over, what did not, what still has to be verified.
|
||||
kvb-check-confidence skill and its kvb-check-score block: what came over, what
|
||||
did not, what still has to be verified.
|
||||
Run the project's test or quality gate only when the operator asks for it.
|
||||
|
||||
## Rules
|
||||
|
||||
@@ -14,18 +14,20 @@ confirms it.
|
||||
1. **Check**: the branch is committed and clean, branched from the default
|
||||
branch (`gh repo view --json defaultBranchRef`), and has no open pull
|
||||
request yet (`gh pr list --head <branch>`).
|
||||
2. **Assess** the branch with the `kvb-check-confidence` skill: fix
|
||||
confidence, break risk, residuals.
|
||||
2. **Assess** the branch with the `kvb-check-confidence` skill: its
|
||||
`kvb-check-score` block and residuals.
|
||||
3. **Push**: hand the operator `gipsn` in the worktree through the
|
||||
`kvb-org-operator` skill, and continue once `git ls-remote --heads origin
|
||||
<branch>` shows the branch's head commit.
|
||||
4. **Body**, written to a scratch file: what changed and why, the test
|
||||
evidence, the confidence block, and one `Closes #<n>` line per issue the
|
||||
evidence, the score block, and one `Closes #<n>` line per issue the
|
||||
branch fixes, each verified as the `kvb-git-issue-close` skill does.
|
||||
5. **Confirm** with one `AskUserQuestion`: title (Conventional Commits),
|
||||
base, state and labels. The state follows the bar of the `kvb-org-split`
|
||||
skill: ready at or above it, a draft with the residuals in the body below
|
||||
it.
|
||||
skill (fix confidence ≥ 90 %, break risk ≤ 5 %, efficiency and speed
|
||||
≤ 100 %): ready at or above it, a draft with the residuals in the body
|
||||
below it. An efficiency or speed over 100 % goes into this question with
|
||||
its cause.
|
||||
6. **Create**: `gh pr create --base <default> --head <branch> --title
|
||||
"<title>" --body-file <file>`, plus `--draft` below the bar.
|
||||
7. **Label** with the `kvb-git-pr-label` skill.
|
||||
|
||||
@@ -12,10 +12,11 @@ holds:
|
||||
failure on it, a red default-branch run or an issue that reproduces there;
|
||||
name the run or issue.
|
||||
- **`90/5`**: the pull request meets the bar of the `kvb-org-split` skill (fix
|
||||
confidence of at least 90 %, break risk under 5 %), from the
|
||||
`kvb-check-confidence` numbers in its body or freshly assessed, and only
|
||||
when `gate.py status` of the `kvb-check-gate` skill prints `gate: green`
|
||||
with the PR's exact head checked out.
|
||||
confidence ≥ 90 %, break risk ≤ 5 %, efficiency and speed ≤ 100 %; over
|
||||
100 %, ask the operator first, naming the cause), from the
|
||||
`kvb-check-score` block in its body or freshly assessed, and only when
|
||||
`gate.py status` of the `kvb-check-gate` skill prints `gate: green` with
|
||||
the PR's exact head checked out.
|
||||
|
||||
## Procedure
|
||||
|
||||
@@ -25,7 +26,8 @@ holds:
|
||||
2. Judge both labels with their evidence.
|
||||
3. A missing label is created once: `gh label create hotfix --color B60205
|
||||
--description "Fixes a failure on the default branch"`, `gh label create
|
||||
90/5 --color 0E8A16 --description "Fix confidence >= 90 %, break risk < 5 %"`.
|
||||
90/5 --color 0E8A16 --description
|
||||
"Fix confidence >= 90 %, break risk <= 5 %, efficiency and speed <= 100 %"`.
|
||||
4. Standalone, confirm the changes with one `AskUserQuestion`; from
|
||||
`kvb-git-pr-create`, its confirmation covers them.
|
||||
5. `gh pr edit <pr> --add-label <l>` or `--remove-label <l>`. A draft that
|
||||
|
||||
@@ -35,9 +35,10 @@ request, owner and lag, nothing preselected. Only the picks are updated.
|
||||
to the operator.
|
||||
- **Any other branch**: in the worktree that already has it checked out,
|
||||
else a temporary one under `/tmp` (the repository's own worktree targets
|
||||
where it has them), `git merge --no-verify --no-edit origin/main`. On a
|
||||
conflict, `git merge --abort` and report the files; the `kvb-git-merge-fix`
|
||||
skill or the operator resolves it later.
|
||||
where it has them), `git merge --no-verify --no-edit origin/main`, with the
|
||||
approval-prompt duty of `kvb-git-commit-no-verify`. On a conflict, `git
|
||||
merge --abort` and report the files; the `kvb-git-merge-fix` skill or the
|
||||
operator resolves it later.
|
||||
- Remove the temporary worktrees afterwards.
|
||||
|
||||
## Step 4: hand over the pushes
|
||||
|
||||
@@ -1,9 +1,8 @@
|
||||
---
|
||||
name: kvb-help
|
||||
description: >
|
||||
List and explain every skill in the session. Trigger on "welche skills
|
||||
gibt es", "skill übersicht", or when the operator asks what
|
||||
skills exist or what one does.
|
||||
List and explain every session skill. Trigger on "welche skills gibt es",
|
||||
"skill übersicht", or questions on what skills exist or what one does.
|
||||
---
|
||||
|
||||
Produce a one-shot overview of every skill the session can invoke:
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
name: kvb-org-coordinate
|
||||
description: >
|
||||
Coordinate a failure or fix with the agents on related branches through
|
||||
Coordinate a failure or fix with agents on related branches through
|
||||
merges: claim, ask, report. Trigger when a fix touches code other
|
||||
branches share.
|
||||
---
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
---
|
||||
name: kvb-org-matrix
|
||||
description: >
|
||||
One matrix of everything in flight with confidence and break risk per
|
||||
item. Trigger on "status", "übersicht", "wo stehen wir".
|
||||
One matrix of everything in flight with the kvb-check-score line per item.
|
||||
Trigger on "status", "übersicht", "wo stehen wir".
|
||||
---
|
||||
|
||||
One-shot overview, no side effects: it reads state and reports it, never
|
||||
@@ -33,14 +33,14 @@ you could not read:
|
||||
|
||||
One table, most-blocking first, one row per item:
|
||||
|
||||
| Item | State | Conf. | Break risk | Evidence / next step |
|
||||
| Item | State | Score | Evidence / next step |
|
||||
|
||||
- **State**: `open`, `running`, `blocked`, `done`. `done` means verified (a
|
||||
passing run, a green check, a reproduction that now succeeds); work that
|
||||
merely looks finished is `open`.
|
||||
- **Conf.** and **Break risk**: two separate percentages per the
|
||||
`kvb-check-confidence` skill, only for rows that changed something; `-`
|
||||
for pure observation rows.
|
||||
- **Score**: the `kvb-check-score` line (🎯 fix, 💥 break, 🌱 efficiency,
|
||||
⚡ speed, 🏆 score) per the `kvb-check-confidence` skill, only for rows
|
||||
that changed something; `-` for pure observation rows.
|
||||
- **Evidence / next step**: for `done` the strongest proof line (what ran,
|
||||
what it returned), otherwise the single next action.
|
||||
|
||||
@@ -58,6 +58,6 @@ Then at most three lines:
|
||||
a forgotten background job or an unpushed commit is what the matrix exists
|
||||
to surface.
|
||||
- Report what the poll returned, never a predicted result; a running item
|
||||
has no confidence number yet.
|
||||
has no score yet.
|
||||
- One row per item, no per-item paragraphs; the operator asks when a row
|
||||
needs explaining.
|
||||
@@ -2,8 +2,8 @@
|
||||
name: kvb-org-perform
|
||||
description: >
|
||||
Pick `model` and `effort` per subagent from a fixed table, cheaper only at
|
||||
90 % confidence of an equal result. Apply AUTOMATICALLY before every Agent
|
||||
tool call.
|
||||
90 % confidence of equal results. Apply AUTOMATICALLY before every Agent
|
||||
call.
|
||||
---
|
||||
|
||||
A subagent runs on the main model and effort unless the spawn names others.
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
---
|
||||
name: kvb-org-robot
|
||||
description: >
|
||||
Work fully autonomously to a goal: no check-ins, no stopping before the goal,
|
||||
only commands already cleared. Trigger when the operator wants hands-off,
|
||||
Work fully autonomously to a goal: no check-ins, no stop before it, only
|
||||
commands already cleared. Trigger when the operator wants hands-off,
|
||||
uninterrupted execution.
|
||||
---
|
||||
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
---
|
||||
name: kvb-org-split
|
||||
description: >
|
||||
Keep agent work in its own worktree until 90% fix confidence and under 5%
|
||||
break risk, else a proposal. Trigger when delegating or parallelizing.
|
||||
Keep agent work in its own worktree until it meets the 90/5 bar
|
||||
(kvb-check-score), else a proposal. Trigger when delegating or parallelizing.
|
||||
---
|
||||
|
||||
# Funnel
|
||||
@@ -12,9 +12,9 @@ in a sidetree until it does, or leaves as a proposal.
|
||||
|
||||
## The bar
|
||||
|
||||
- **Fix confidence of at least 90 %** and **break risk under 5 %**, both
|
||||
reported by the `kvb-check-confidence` skill, the break risk derived through
|
||||
the `kvb-check-break-risk` skill.
|
||||
- **Fix confidence ≥ 90 %, break risk ≤ 5 %, efficiency and speed ≤ 100 %**,
|
||||
all from the `kvb-check-score` block of `kvb-check-confidence`; over 100 %,
|
||||
ask the operator before the commit, naming the cause.
|
||||
- The operator may set a different bar. Never lower it to make a change fit.
|
||||
|
||||
## The stages
|
||||
@@ -45,8 +45,8 @@ in a sidetree until it does, or leaves as a proposal.
|
||||
3. **Evidence.** Targeted tests, a test for the new behaviour, and a negative
|
||||
control: the test fails without the change. Numbers come from the same
|
||||
source before and after.
|
||||
4. **Self-assessment.** The agent reports fix confidence and break risk with
|
||||
their residuals and commits on its branch. Its parent marks proven key
|
||||
4. **Self-assessment.** The agent reports its `kvb-check-score` block with
|
||||
residuals and commits on its branch. Its parent marks proven key
|
||||
results met in the `kvb-org-rasci` register and records
|
||||
`--artifact worktree:<path>`.
|
||||
5. **Independent review.** Before any merge, run the `kvb-think-dialectic`
|
||||
@@ -79,7 +79,7 @@ inherently riskier, read `proposal.md` next to this file.
|
||||
|
||||
One table per round:
|
||||
|
||||
| Agent | Sidetree branch | Confidence | Break risk | Verdict | Evidence |
|
||||
| Agent | Sidetree branch | Score | Verdict | Evidence |
|
||||
|
||||
Verdict: merged, proposal or abandoned; proposals list what would close their
|
||||
gap.
|
||||
@@ -5,7 +5,10 @@ production-like dataset or an upstream answer, is not available, or the change
|
||||
is inherently riskier), the agent stops trying and submits a proposal:
|
||||
|
||||
- what the change does and why it is worth having;
|
||||
- the confidence and break risk it reached, with their residuals;
|
||||
- the `kvb-check-score` block it reached, with residuals: which part of the
|
||||
bar it misses (fix confidence ≥ 90 %, break risk ≤ 5 %, efficiency and
|
||||
speed ≤ 100 %), and the cause of any efficiency or speed over 100 %, which
|
||||
the operator weighs before the commit;
|
||||
- exactly which evidence would close the gap, and what obtaining it costs;
|
||||
- where the work lives (branch and commit), so nothing has to be redone;
|
||||
- a recommendation: apply after that evidence, apply now with a named risk, or
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
name: kvb-skill-auto
|
||||
description: >
|
||||
Turn operator repetition into a skill. Trigger when the operator repeats an
|
||||
instruction, correction or prompt pattern a third time, or says they keep
|
||||
instruction, correction or prompt pattern thrice, or says they keep
|
||||
repeating themselves.
|
||||
---
|
||||
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
---
|
||||
name: kvb-think-criticize
|
||||
description: >
|
||||
Constructive criticism of a plan, design, decision, text or code, each
|
||||
point hardened by dialectic. Trigger on "kritisiere",
|
||||
Constructive criticism of a plan, design, decision, text or code,
|
||||
hardened by dialectic. Trigger on "kritisiere",
|
||||
"konstruktive Kritik", "was haeltst du davon", or requests for honest
|
||||
feedback.
|
||||
---
|
||||
@@ -79,8 +79,8 @@ One line of scope, then the steelman. Then:
|
||||
`Evidence` is `proven` (read, run, reproduced, cited) or `derived` (reasoned).
|
||||
|
||||
Close with at most three lines: how many points dialectic dropped and why, the
|
||||
single change that helps most, and the `kvb-check-confidence` skill's numbers
|
||||
for the whole critique.
|
||||
single change that helps most, and the `kvb-check-score` block for the whole
|
||||
critique.
|
||||
|
||||
## Rules
|
||||
|
||||
|
||||
@@ -2,8 +2,8 @@
|
||||
name: kvb-think-dialectic
|
||||
description: >
|
||||
Thesis, independent skeptic antithesis and synthesis, iterated to ~99%
|
||||
confidence. Trigger for any root cause, design or decision that is
|
||||
uncertain and expensive to get wrong.
|
||||
confidence. Trigger for any uncertain root cause, design or decision
|
||||
costly to get wrong.
|
||||
---
|
||||
|
||||
Reach the truth by dialectic, not by first guess. Do not stop at a plausible
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
---
|
||||
name: kvb-think-gaps
|
||||
description: >
|
||||
Find and rank missing work (structural flaws, missing features); each pick
|
||||
goes to kvb-create-requirement. Trigger on "was fehlt noch" or when asked
|
||||
Find and rank missing work (structural flaws, missing features); picks go
|
||||
to kvb-create-requirement. Trigger on "was fehlt noch" or when asked
|
||||
what to build next.
|
||||
---
|
||||
|
||||
@@ -79,5 +79,5 @@ shapes go into its Decisions section.
|
||||
- Findings only: no file, implementation, refactor or staging in this skill.
|
||||
- Every candidate traces to evidence: a `file:line`, a failing path, a
|
||||
missing artifact. "Would be nice" is dropped.
|
||||
- Close with the `kvb-check-confidence` skill's numbers for the analysis: how
|
||||
- Close with the `kvb-check-score` block for the analysis: how
|
||||
sure the gaps are real, and what the passes did not cover.
|
||||
@@ -1,9 +1,9 @@
|
||||
---
|
||||
name: kvb-think-listen
|
||||
description: >
|
||||
Ask focused questions until every requirement is pinned down, then reflect
|
||||
the task back. Trigger on unclear scope, constraints or success criteria,
|
||||
or when a skill needs an operator-only fact.
|
||||
Pin down every requirement by focused questions, then reflect the task
|
||||
back. Trigger on unclear scope, constraints or success criteria, or a skill
|
||||
needing an operator-only fact.
|
||||
---
|
||||
|
||||
Do not act on a half-understood request. Listen actively until you could state
|
||||
|
||||
@@ -0,0 +1,264 @@
|
||||
"""Exercise kvb-check-efficiency: tokens.py on skill text changes, measure.py on a command in a base worktree."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import contextlib
|
||||
import importlib.util
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest import mock
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parents[1]
|
||||
SCRIPTS = REPO_ROOT / "skills" / "kvb-check-efficiency" / "scripts"
|
||||
TOKENS = SCRIPTS / "tokens.py"
|
||||
SPEC = importlib.util.spec_from_file_location("measure", SCRIPTS / "measure.py")
|
||||
measure = importlib.util.module_from_spec(SPEC)
|
||||
SPEC.loader.exec_module(measure)
|
||||
BUDGET = REPO_ROOT / "scripts" / "budget.py"
|
||||
SKILL_A = "---\nname: kvb-a\ndescription: >\n one two three\n---\n"
|
||||
SKILL_B = "---\nname: kvb-b\ndescription: >\n four five\n---\n"
|
||||
CHEAP = "print('cheap')\n"
|
||||
COSTLY = "data = list(range(2_000_000))\nprint(sum(data))\n"
|
||||
LINE = re.compile(
|
||||
r"^🌱 efficiency (?P<mark>~?)(?P<efficiency>\d+) % \(cpu \d+\.\d\d→\d+\.\d\d s, ram \d+\.\d→\d+\.\d MB\) · "
|
||||
r"⚡ speed (?P<speed_mark>~?)(?P<speed>\d+) % \((?P<basis>wall|cpu) \d+\.\d\d→\d+\.\d\d s\)(?P<tail>.*)$"
|
||||
)
|
||||
|
||||
|
||||
class GitRepo(unittest.TestCase):
|
||||
def setUp(self):
|
||||
tmp = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(tmp.cleanup)
|
||||
self.base = Path(tmp.name)
|
||||
self.repo = self.base / "repo"
|
||||
self.repo.mkdir()
|
||||
self.env = {
|
||||
"PATH": os.environ["PATH"],
|
||||
"HOME": str(self.base),
|
||||
"GIT_CONFIG_NOSYSTEM": "1",
|
||||
"GIT_AUTHOR_NAME": "Test",
|
||||
"GIT_AUTHOR_EMAIL": "test@example.org",
|
||||
"GIT_COMMITTER_NAME": "Test",
|
||||
"GIT_COMMITTER_EMAIL": "test@example.org",
|
||||
}
|
||||
self.git("init", "-q", "-b", "main")
|
||||
|
||||
def git(self, *args: str) -> str:
|
||||
return subprocess.run(
|
||||
["git", *args], cwd=self.repo, env=self.env, capture_output=True, text=True, check=True
|
||||
).stdout
|
||||
|
||||
def write(self, path: str, text: str) -> None:
|
||||
file = self.repo / path
|
||||
file.parent.mkdir(parents=True, exist_ok=True)
|
||||
file.write_text(text, encoding="utf-8")
|
||||
|
||||
def commit(self) -> None:
|
||||
self.git("add", "-A")
|
||||
self.git("commit", "-qm", "base")
|
||||
|
||||
def run_script(self, script: Path, *args: str, cwd: Path) -> subprocess.CompletedProcess:
|
||||
return subprocess.run(
|
||||
[sys.executable, str(script), *args], cwd=cwd, env=self.env, capture_output=True, text=True, check=False
|
||||
)
|
||||
|
||||
|
||||
class TestTokens(GitRepo):
|
||||
def setUp(self):
|
||||
super().setUp()
|
||||
self.write("scripts/budget.py", BUDGET.read_text(encoding="utf-8"))
|
||||
self.write("skills/kvb-a/SKILL.md", SKILL_A)
|
||||
self.write("skills/kvb-a/notes.md", "x" * 400)
|
||||
self.write("skills/kvb-a/scripts/tool.py", "print(1)\n")
|
||||
self.commit()
|
||||
|
||||
def tokens(self, *extra: str) -> subprocess.CompletedProcess:
|
||||
return self.run_script(TOKENS, "--repo", str(self.repo), "--base", "HEAD", *extra, cwd=self.base)
|
||||
|
||||
def test_shrunk_file_is_under_100(self):
|
||||
self.write("skills/kvb-a/notes.md", "x" * 200)
|
||||
self.write("skills/kvb-a/scripts/tool.py", "print(1)\n" * 50)
|
||||
result = self.tokens()
|
||||
self.assertEqual(result.stdout, "🌱 efficiency 50 % (skills 100→50 tokens; listing 3→3 words)\n")
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
|
||||
def test_changed_and_new_skill_files_over_100(self):
|
||||
self.write("skills/kvb-a/notes.md", "x" * 600)
|
||||
self.write("skills/kvb-b/SKILL.md", SKILL_B.ljust(200, "y"))
|
||||
result = self.tokens()
|
||||
self.assertEqual(result.stdout, "🌱 efficiency 200 % (skills 100→200 tokens; listing 3→5 words)\n")
|
||||
self.assertEqual(result.returncode, 3)
|
||||
result = self.tokens("--json")
|
||||
self.assertEqual(result.returncode, 3)
|
||||
self.assertEqual(
|
||||
json.loads(result.stdout),
|
||||
{
|
||||
"efficiency": 200,
|
||||
"skills": {"before": 100, "after": 200},
|
||||
"listing": {"before": 3, "after": 5},
|
||||
"per_skill": {"kvb-a": {"before": 100, "after": 150}, "kvb-b": {"before": 0, "after": 50}},
|
||||
},
|
||||
)
|
||||
|
||||
def test_only_new_files_print_new(self):
|
||||
self.write("skills/kvb-b/SKILL.md", SKILL_B.ljust(200, "y"))
|
||||
result = self.tokens()
|
||||
self.assertEqual(result.stdout, "🌱 efficiency new (+50 tokens; listing 3→5 words)\n")
|
||||
self.assertEqual(result.returncode, 3)
|
||||
self.assertIsNone(json.loads(self.tokens("--json").stdout)["efficiency"])
|
||||
|
||||
def test_rename_counts_both_sides(self):
|
||||
self.git("mv", "skills/kvb-a/notes.md", "skills/kvb-a/moved.md")
|
||||
result = self.tokens()
|
||||
self.assertEqual(result.stdout, "🌱 efficiency 100 % (skills 100→100 tokens; listing 3→3 words)\n")
|
||||
self.assertEqual(result.returncode, 0)
|
||||
|
||||
def test_no_change_is_100(self):
|
||||
result = self.tokens()
|
||||
self.assertEqual(result.stdout, "🌱 efficiency 100 % (skills 0→0 tokens; listing 3→3 words)\n")
|
||||
self.assertEqual(result.returncode, 0)
|
||||
|
||||
def test_usage_errors_exit_2(self):
|
||||
self.assertEqual(self.run_script(TOKENS, "--repo", str(self.repo), "--base", "nope", cwd=self.base).returncode, 2)
|
||||
self.assertEqual(self.run_script(TOKENS, "--repo", str(self.base), "--base", "HEAD", cwd=self.base).returncode, 2)
|
||||
(self.repo / "scripts" / "budget.py").unlink()
|
||||
self.assertEqual(self.tokens().returncode, 2)
|
||||
self.assertEqual(self.run_script(TOKENS, "--repo", str(self.repo), cwd=self.base).returncode, 2)
|
||||
|
||||
|
||||
class Ticks:
|
||||
def __init__(self, busy: int):
|
||||
self.busy = busy
|
||||
self.calls = 0
|
||||
|
||||
def __call__(self) -> tuple[int, int]:
|
||||
self.calls += 1
|
||||
return self.calls * self.busy, self.calls * 100
|
||||
|
||||
|
||||
class TestMeasure(GitRepo):
|
||||
def setUp(self):
|
||||
super().setUp()
|
||||
self.write("pkg/work.py", COSTLY)
|
||||
self.commit()
|
||||
self.leftovers = set(Path("/tmp").glob("kvb-measure-*"))
|
||||
|
||||
def measured(self, *args: str, ticks=None, cwd: Path | None = None) -> tuple[int, str, str, int]:
|
||||
out, err = io.StringIO(), io.StringIO()
|
||||
with (
|
||||
mock.patch.dict(os.environ, self.env, clear=True),
|
||||
mock.patch.object(measure, "cpu_ticks", ticks or Ticks(0)),
|
||||
mock.patch.object(measure, "SAMPLE_SECONDS", 0),
|
||||
mock.patch.object(measure, "sample", wraps=measure.sample) as samples,
|
||||
contextlib.chdir(cwd or self.repo / "pkg"),
|
||||
contextlib.redirect_stdout(out),
|
||||
contextlib.redirect_stderr(err),
|
||||
):
|
||||
try:
|
||||
code = measure.main(list(args))
|
||||
except SystemExit as stop:
|
||||
code = stop.code
|
||||
return code, out.getvalue(), err.getvalue(), samples.call_count
|
||||
|
||||
def line(self, out: str) -> re.Match:
|
||||
match = LINE.match(out.strip())
|
||||
self.assertIsNotNone(match, out)
|
||||
return match
|
||||
|
||||
def assertCleanedUp(self):
|
||||
trees = [line for line in self.git("worktree", "list", "--porcelain").splitlines() if line.startswith("worktree ")]
|
||||
self.assertEqual(trees, [f"worktree {self.repo.resolve()}"])
|
||||
self.assertLessEqual(set(Path("/tmp").glob("kvb-measure-*")), self.leftovers)
|
||||
|
||||
def test_cheaper_head_stops_after_one_run_each(self):
|
||||
self.write("pkg/work.py", CHEAP)
|
||||
code, out, err, samples = self.measured("--base", "HEAD", "--", sys.executable, "work.py")
|
||||
self.assertEqual((code, samples), (0, 2), err)
|
||||
match = self.line(out)
|
||||
self.assertEqual((match["mark"], match["speed_mark"], match["basis"], match["tail"]), ("", "", "wall", ""))
|
||||
self.assertLess(int(match["efficiency"]), 100)
|
||||
self.assertLess(int(match["speed"]), 100)
|
||||
self.assertCleanedUp()
|
||||
|
||||
def test_costlier_head_takes_the_mean_of_5_and_exits_3(self):
|
||||
self.write("pkg/work.py", CHEAP)
|
||||
self.commit()
|
||||
self.write("pkg/work.py", COSTLY)
|
||||
code, out, err, samples = self.measured("--base", "HEAD", "--", sys.executable, "work.py")
|
||||
self.assertEqual((code, samples), (3, 10), err)
|
||||
match = self.line(out)
|
||||
self.assertEqual((match["basis"], match["tail"]), ("wall", " · mean of 5"))
|
||||
self.assertGreater(int(match["efficiency"]), 100)
|
||||
self.assertGreater(int(match["speed"]), 100)
|
||||
code, out, _, _ = self.measured("--base", "HEAD", "--json", "--", sys.executable, "work.py")
|
||||
found = json.loads(out)
|
||||
self.assertEqual((code, found["runs"], found["estimated"], found["other_load"]), (3, 5, False, 0))
|
||||
self.assertGreater(found["head"]["ram"], found["base"]["ram"])
|
||||
self.assertCleanedUp()
|
||||
|
||||
def test_other_load_over_30_makes_an_estimate_from_cpu_time(self):
|
||||
self.write("pkg/work.py", CHEAP)
|
||||
code, out, err, samples = self.measured("--base", "HEAD", "--", sys.executable, "work.py", ticks=Ticks(50))
|
||||
self.assertEqual((code, samples), (0, 2), err)
|
||||
match = self.line(out)
|
||||
self.assertEqual(
|
||||
(match["mark"], match["speed_mark"], match["basis"], match["tail"]),
|
||||
("~", "~", "cpu", " (estimated: other load 50 %)"),
|
||||
)
|
||||
_, out, _, _ = self.measured("--base", "HEAD", "--json", "--", sys.executable, "work.py", ticks=Ticks(50))
|
||||
found = json.loads(out)
|
||||
self.assertEqual((found["estimated"], found["other_load"]), (True, 50))
|
||||
self.assertEqual(found["speed"], round(measure.ratio(found["head"]["cpu"], found["base"]["cpu"]) * 100))
|
||||
_, out, _, _ = self.measured("--base", "HEAD", "--json", "--", sys.executable, "work.py", ticks=Ticks(30))
|
||||
self.assertEqual({k: json.loads(out)[k] for k in ("estimated", "other_load")}, {"estimated": False, "other_load": 30})
|
||||
self.assertCleanedUp()
|
||||
|
||||
def test_unreadable_load_is_an_estimate(self):
|
||||
self.write("pkg/work.py", CHEAP)
|
||||
code, out, err, _ = self.measured("--base", "HEAD", "--", sys.executable, "work.py", ticks=lambda: None)
|
||||
self.assertEqual(code, 0, err)
|
||||
self.assertEqual(self.line(out)["tail"], " (estimated: other load unknown)")
|
||||
|
||||
@unittest.skipUnless(measure.PROC_STAT.is_file(), "no /proc/stat on this platform")
|
||||
def test_proc_stat_yields_busy_and_total_jiffies(self):
|
||||
busy, total = measure.cpu_ticks()
|
||||
self.assertLessEqual(0, busy)
|
||||
self.assertLessEqual(busy, total)
|
||||
|
||||
def test_failing_command_names_the_tree_and_exits_2(self):
|
||||
self.write("pkg/work.py", "import sys\nsys.exit('broken here')\n")
|
||||
code, out, err, _ = self.measured("--base", "HEAD", "--", sys.executable, "work.py")
|
||||
self.assertEqual((code, out), (2, ""))
|
||||
self.assertIn("exited 1 in the head tree", err)
|
||||
self.assertIn("broken here", err)
|
||||
self.assertCleanedUp()
|
||||
|
||||
def test_usage_errors_exit_2(self):
|
||||
for args in (
|
||||
("--base", "HEAD", sys.executable, "work.py"),
|
||||
("--base", "HEAD", "--"),
|
||||
("--base", "HEAD", "--runs", "1", "--", sys.executable, "work.py"),
|
||||
("--base", "nope", "--", sys.executable, "work.py"),
|
||||
("--", sys.executable, "work.py"),
|
||||
):
|
||||
with self.subTest(args=args):
|
||||
self.assertEqual(self.measured(*args)[0], 2)
|
||||
self.assertCleanedUp()
|
||||
|
||||
def test_missing_directory_at_base_exits_2(self):
|
||||
self.write("new/work.py", CHEAP)
|
||||
code, _, err, _ = self.measured("--base", "HEAD", "--", sys.executable, "work.py", cwd=self.repo / "new")
|
||||
self.assertEqual(code, 2)
|
||||
self.assertIn("new does not exist at HEAD", err)
|
||||
self.assertCleanedUp()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,104 @@
|
||||
"""Exercise score.py of kvb-check-score: formula, dash and new values, capping, every gate verdict and exit code."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
import sys
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parents[1]
|
||||
SCORE = REPO_ROOT / "skills" / "kvb-check-score" / "scripts" / "score.py"
|
||||
PASS = "✅ gate: 90/5, efficiency and speed <= 100"
|
||||
|
||||
|
||||
class TestScore(unittest.TestCase):
|
||||
def _run(self, *args: str) -> subprocess.CompletedProcess:
|
||||
return subprocess.run([sys.executable, str(SCORE), *args], capture_output=True, text=True, check=False)
|
||||
|
||||
def _score(self, fix: str, risk: str, efficiency: str, speed: str) -> tuple[list[str], int]:
|
||||
result = self._run("--fix", fix, "--break", risk, "--efficiency", efficiency, "--speed", speed)
|
||||
self.assertEqual(result.stderr, "")
|
||||
return result.stdout.splitlines(), result.returncode
|
||||
|
||||
def test_formula_and_line_format(self):
|
||||
self.assertEqual(
|
||||
self._score("95", "4", "100", "100"),
|
||||
(["🎯 fix 95 % · 💥 break 4 % · 🌱 efficiency 100 % · ⚡ speed 100 % · 🏆 score 91", PASS], 0),
|
||||
)
|
||||
lines, code = self._score("95", "2", "97", "-")
|
||||
self.assertEqual(lines[0], "🎯 fix 95 % · 💥 break 2 % · 🌱 efficiency 97 % · ⚡ speed – · 🏆 score 96")
|
||||
self.assertEqual(code, 0)
|
||||
self.assertEqual(self._score("92.5", "0", "100", "100")[0][0].split(" · ")[0], "🎯 fix 92.5 %")
|
||||
|
||||
def test_dash_values_count_as_100(self):
|
||||
self.assertEqual(
|
||||
self._score("92", "3", "-", "-"),
|
||||
(["🎯 fix 92 % · 💥 break 3 % · 🌱 efficiency – · ⚡ speed – · 🏆 score 89", PASS], 0),
|
||||
)
|
||||
|
||||
def test_score_rounds_half_up(self):
|
||||
for args, total in (
|
||||
(("94.5", "0", "-", "-"), 95),
|
||||
(("90", "5", "100", "100"), 86),
|
||||
(("91", "5", "-", "-"), 86),
|
||||
(("97", "0", "97", "-"), 100),
|
||||
):
|
||||
with self.subTest(args=args):
|
||||
self.assertTrue(self._score(*args)[0][0].endswith(f"🏆 score {total}"))
|
||||
|
||||
def test_score_is_capped(self):
|
||||
lines, code = self._score("100", "0", "50", "50")
|
||||
self.assertTrue(lines[0].endswith("🏆 score 100"))
|
||||
self.assertEqual((lines[1], code), (PASS, 0))
|
||||
lines, code = self._score("0", "100", "-", "-")
|
||||
self.assertTrue(lines[0].endswith("🏆 score 0"))
|
||||
self.assertEqual(code, 1)
|
||||
|
||||
def test_boundaries_pass(self):
|
||||
self.assertEqual(self._score("90", "5", "100", "100")[1], 0)
|
||||
|
||||
def test_90_5_failures_exit_1(self):
|
||||
for args, verdict in (
|
||||
(("89.5", "1", "-", "-"), "⛔ gate: fix 89.5 % < 90"),
|
||||
(("95", "6", "-", "-"), "⛔ gate: break 6 % > 5"),
|
||||
(("80", "10", "-", "-"), "⛔ gate: fix 80 % < 90, break 10 % > 5"),
|
||||
(("85", "2", "150", "new"), "⛔ gate: fix 85 % < 90"),
|
||||
):
|
||||
with self.subTest(args=args):
|
||||
lines, code = self._score(*args)
|
||||
self.assertEqual((lines[1], code), (verdict, 1))
|
||||
|
||||
def test_cost_over_100_asks_the_operator_exit_3(self):
|
||||
cases = {
|
||||
("120", "100"): "⚠️ gate: efficiency 120 % > 100: ask the operator before the commit",
|
||||
("-", "130"): "⚠️ gate: speed 130 % > 100: ask the operator before the commit",
|
||||
("120", "130"): "⚠️ gate: efficiency 120 % > 100, speed 130 % > 100: ask the operator before the commit",
|
||||
("100.5", "-"): "⚠️ gate: efficiency 100.5 % > 100: ask the operator before the commit",
|
||||
("new", "-"): "⚠️ gate: efficiency new: ask the operator before the commit",
|
||||
}
|
||||
for (efficiency, speed), verdict in cases.items():
|
||||
with self.subTest(efficiency=efficiency, speed=speed):
|
||||
lines, code = self._score("95", "2", efficiency, speed)
|
||||
self.assertEqual((lines[1], code), (verdict, 3))
|
||||
|
||||
def test_new_counts_as_100_in_the_score(self):
|
||||
lines, _ = self._score("95", "0", "new", "-")
|
||||
self.assertEqual(lines[0], "🎯 fix 95 % · 💥 break 0 % · 🌱 efficiency new · ⚡ speed – · 🏆 score 95")
|
||||
|
||||
def test_usage_errors_exit_2(self):
|
||||
good = {"--fix": "95", "--break": "2", "--efficiency": "100", "--speed": "100"}
|
||||
bad = (
|
||||
("--fix", "101"), ("--fix", "-1"), ("--fix", "nan"), ("--break", "x"), ("--break", "-"),
|
||||
("--efficiency", "0"), ("--efficiency", "-5"), ("--efficiency", "abc"), ("--speed", "inf"),
|
||||
)
|
||||
for flag, value in bad:
|
||||
with self.subTest(flag=flag, value=value):
|
||||
args = [item for key, val in {**good, flag: value}.items() for item in (key, val)]
|
||||
result = self._run(*args)
|
||||
self.assertEqual((result.returncode, result.stdout), (2, ""))
|
||||
self.assertEqual(self._run("--fix", "95", "--break", "2", "--efficiency", "100").returncode, 2)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
Reference in new issue
Block a user