feat(kvb-skill-silence): cut what skills cost in tokens and messages
The new skill measures before it changes anything (listing and SKILL.md sizes, hook output, messages and news, subagent runs, context re-read per turn) and works seven levers in order: listing, length, messages by the Peer protocol, subagents, graph first, hooks and the context window; every change waits for the operator's pick, and another repository's skills get a register task instead of a write. kvb-optimize runs it right after kvb-skill-essence. context.py reports per UTC day, from the transcripts Claude Code keeps, the turns (one per message id, although the transcript repeats usage per content block), context tokens, median, p90, compactions and the shares of turns and tokens above each threshold, so no cron is needed. enable-plugins.js sets CLAUDE_CODE_AUTO_COMPACT_WINDOW to 500000 only where it is unset, as the operator's trial value; silence proposes a new one only with the series as evidence, never on its own. test_context and test_dream cover both. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
97e239a67c
commit
c5497ebadc
7 files changed
+265
-5
No files matched your search
@@ -86,6 +86,7 @@ flowchart LR
|
||||
skill --> skill_transcend[kvb-skill-transcend]
|
||||
skill --> skill_compile[kvb-skill-compile]
|
||||
skill --> skill_cleanup[kvb-skill-cleanup]
|
||||
skill --> skill_silence[kvb-skill-silence]
|
||||
none --> cleanup[kvb-cleanup]
|
||||
none --> help[kvb-help]
|
||||
none --> transcend_all[kvb-transcend]
|
||||
@@ -184,6 +185,7 @@ flowchart LR
|
||||
skill_transcend[kvb-skill-transcend]
|
||||
compile[kvb-skill-compile]
|
||||
skill_cleanup[kvb-skill-cleanup]
|
||||
silence[kvb-skill-silence]
|
||||
end
|
||||
subgraph cat_none [no category]
|
||||
cleanup[kvb-cleanup]
|
||||
@@ -289,6 +291,8 @@ flowchart LR
|
||||
optimize --> essence
|
||||
optimize --> refactor
|
||||
optimize --> tune
|
||||
optimize --> silence
|
||||
silence -. peer protocol .-> democracy
|
||||
optimize --> agent_optimize
|
||||
optimize --> gaps
|
||||
agent_optimize -. install .-> operator
|
||||
|
||||
@@ -48,6 +48,8 @@ for (const [name, entry] of Object.entries(MARKETPLACES)) {
|
||||
if (!data.extraKnownMarketplaces[name]) data.extraKnownMarketplaces[name] = entry;
|
||||
}
|
||||
data.enabledPlugins = Object.assign(data.enabledPlugins || {}, PLUGINS);
|
||||
data.env = data.env || {};
|
||||
if (!data.env.CLAUDE_CODE_AUTO_COMPACT_WINDOW) data.env.CLAUDE_CODE_AUTO_COMPACT_WINDOW = "500000";
|
||||
|
||||
const pluginRepos = new Set(Object.values(MARKETPLACES).map((entry) => entry.source.repo));
|
||||
const lock = JSON.parse(fs.readFileSync(path.join(__dirname, "..", "skills-lock.json"), "utf8"));
|
||||
|
||||
@@ -23,12 +23,14 @@ inventory is skipped with one line.
|
||||
4. `kvb-skill-tune`: turn frictions into new or sharper skills.
|
||||
5. `kvb-skill-compile`: turn deterministic skill steps into scripts.
|
||||
6. `kvb-skill-essence`: compact skills and memory.
|
||||
7. `kvb-code-refactor`: bring the uncommitted changes up to the project's
|
||||
7. `kvb-skill-silence`: cut the tokens and agent messages the skills still
|
||||
cost.
|
||||
8. `kvb-code-refactor`: bring the uncommitted changes up to the project's
|
||||
standards.
|
||||
8. `kvb-code-tune`: optimize the hot paths the evidence measured.
|
||||
9. `kvb-agent-optimize`: index every repository in scope as a code graph
|
||||
and tighten its agent instructions.
|
||||
10. `kvb-think-gaps`: find the missing work and hand each pick to
|
||||
9. `kvb-code-tune`: optimize the hot paths the evidence measured.
|
||||
10. `kvb-agent-optimize`: index every repository in scope as a code graph
|
||||
and tighten its agent instructions.
|
||||
11. `kvb-think-gaps`: find the missing work and hand each pick to
|
||||
`kvb-create-requirement`.
|
||||
|
||||
## Questions
|
||||
|
||||
@@ -0,0 +1,66 @@
|
||||
---
|
||||
name: kvb-skill-silence
|
||||
description: >
|
||||
Cut the tokens and agent messages the skills of this repository and of the
|
||||
skills repo cost. kvb-optimize runs it.
|
||||
---
|
||||
|
||||
Every word in the skill listing, a loaded SKILL.md or a hook's output is paid
|
||||
on every turn of every session, and every delivered message costs its
|
||||
recipient a full turn. `kvb-skill-essence` removes dead weight and
|
||||
duplicates; this skill cuts what still costs.
|
||||
|
||||
## Scope and measurement
|
||||
|
||||
The skills the current repository owns (`skills/`, its own `.claude/skills/`,
|
||||
not installed copies) and the skills repository. Measure before changing:
|
||||
|
||||
- `scripts/budget.py --repo <skills repo> --top 10`: listing words, the
|
||||
longest descriptions, every SKILL.md over the cap.
|
||||
- Each own SKILL.md's word count, and what each registered hook prints per
|
||||
turn (settings files, read-only).
|
||||
- The window's traffic: this session's `SendMessage` calls, the news
|
||||
(`news.py show --all` of the `kvb-org-organize` skill), and subagent runs
|
||||
(`log.py report` of the `kvb-org-perform` skill).
|
||||
- Context re-read per turn, from the transcripts Claude Code keeps:
|
||||
`${CLAUDE_SKILL_DIR}/scripts/context.py --root ~/.claude/projects --days 7
|
||||
--threshold 300000 --threshold 500000` (turns, tokens, median, p90,
|
||||
compactions, shares above each threshold, per day).
|
||||
|
||||
## Levers, in this order
|
||||
|
||||
1. **Listing**: a description says what and when, never its own slash
|
||||
command; no skill listed twice through a copy and a plugin.
|
||||
2. **Length**: each SKILL.md at most `SKILL_WORDS` (600) words of
|
||||
`scripts/budget.py`, enforced by `tests/test_lock.py`. Details move to a
|
||||
side file the skill reads only when it needs them; no rule is cut.
|
||||
3. **Messages**: every step that sends one is checked against the
|
||||
`kvb-org-organize` Peer protocol: status that asks nothing goes to
|
||||
`news.py post`, polling peers becomes reading the council, the register or
|
||||
the news, one batched message per peer, no reply to a message that asks
|
||||
nothing, light runs admit themselves.
|
||||
4. **Subagents**: their number follows the stakes, the brief names exact
|
||||
files and claims and caps the answer, a cheaper model only by the
|
||||
`kvb-org-perform` table.
|
||||
5. **Exploration**: code questions go to the code graph first (the
|
||||
`kvb-agent-optimize` skill), grep and reading after.
|
||||
6. **Hooks**: a hook prints nothing when nothing changed; a reminder fires
|
||||
once per window, not per turn.
|
||||
7. **Context window**: `make install` sets `CLAUDE_CODE_AUTO_COMPACT_WINDOW`
|
||||
to 500000 where it is unset. Propose a new value only with the
|
||||
`context.py` series as evidence (tokens saved, compactions, and the
|
||||
operator's corrections of agents that lost a detail after one); the
|
||||
operator picks and changes it, because the settings are the operator's.
|
||||
Never adjust it automatically.
|
||||
|
||||
## Confirm and apply
|
||||
|
||||
One `AskUserQuestion` batch, or the round of the `kvb-optimize` skill: every
|
||||
change with its measured cost before and its estimate after. Only what is
|
||||
picked gets written. A change to another repository's skills becomes a task
|
||||
for that repository's owner in the `kvb-org-rasci` register, never a write
|
||||
from here.
|
||||
|
||||
## Output
|
||||
|
||||
| Repository | Skill or hook | Lever | Before | After |
|
||||
Executable
+117
@@ -0,0 +1,117 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Report how much context agent sessions re-read per turn, from Claude Code's session transcripts.
|
||||
|
||||
Args:
|
||||
--root: transcripts root, e.g. ~/.claude/projects (every */*.jsonl below).
|
||||
--days: how many days back, one row per day, newest last.
|
||||
--threshold: a context size in tokens; per threshold the report adds the
|
||||
share of turns above it and the share of context tokens above it.
|
||||
Repeatable.
|
||||
--json: print the rows as JSON instead of tab-separated lines.
|
||||
|
||||
Output:
|
||||
per day: date, turns, context tokens, median and p90 context per turn,
|
||||
compactions, then per threshold `>N: turns% tokens%`. A turn is one model
|
||||
message, counted once although the transcript repeats its usage per
|
||||
content block.
|
||||
|
||||
Exit codes:
|
||||
0 done, 2 usage error.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import datetime
|
||||
import json
|
||||
import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def fail(message: str) -> None:
|
||||
print(f"context: {message}", file=sys.stderr)
|
||||
sys.exit(2)
|
||||
|
||||
|
||||
def scan(root: Path, since: float) -> tuple[dict[str, tuple[str, int]], dict[str, int]]:
|
||||
turns: dict[str, tuple[str, int]] = {}
|
||||
compactions: dict[str, int] = {}
|
||||
for path in root.glob("*/*.jsonl"):
|
||||
if path.stat().st_mtime < since:
|
||||
continue
|
||||
with path.open(encoding="utf-8", errors="ignore") as handle:
|
||||
for line in handle:
|
||||
if '"usage"' not in line and "compact_boundary" not in line:
|
||||
continue
|
||||
try:
|
||||
entry = json.loads(line)
|
||||
except ValueError:
|
||||
continue
|
||||
day = str(entry.get("timestamp", ""))[:10]
|
||||
if not day:
|
||||
continue
|
||||
if entry.get("subtype") == "compact_boundary":
|
||||
compactions[day] = compactions.get(day, 0) + 1
|
||||
message = entry.get("message")
|
||||
usage = message.get("usage") if isinstance(message, dict) else None
|
||||
if usage and message.get("id"):
|
||||
size = sum(usage.get(k) or 0 for k in ("input_tokens", "cache_read_input_tokens", "cache_creation_input_tokens"))
|
||||
turns[message["id"]] = (day, size)
|
||||
return turns, compactions
|
||||
|
||||
|
||||
def rows(root: Path, days: int, thresholds: list[int], now: float) -> list[dict]:
|
||||
first = datetime.datetime.fromtimestamp(now, datetime.UTC).date() - datetime.timedelta(days=days - 1)
|
||||
start = datetime.datetime.combine(first, datetime.time.min, tzinfo=datetime.UTC).timestamp()
|
||||
turns, compactions = scan(root, start)
|
||||
by_day: dict[str, list[int]] = {}
|
||||
for day, size in turns.values():
|
||||
if day >= first.isoformat():
|
||||
by_day.setdefault(day, []).append(size)
|
||||
out = []
|
||||
for offset in range(days):
|
||||
day = (first + datetime.timedelta(days=offset)).isoformat()
|
||||
sizes = sorted(by_day.get(day, []))
|
||||
total = sum(sizes)
|
||||
row = {
|
||||
"day": day,
|
||||
"turns": len(sizes),
|
||||
"tokens": total,
|
||||
"median": sizes[len(sizes) // 2] if sizes else 0,
|
||||
"p90": sizes[int(len(sizes) * 0.9)] if sizes else 0,
|
||||
"compactions": compactions.get(day, 0),
|
||||
"over": {},
|
||||
}
|
||||
for limit in thresholds:
|
||||
above = [s for s in sizes if s > limit]
|
||||
row["over"][str(limit)] = {
|
||||
"turns": round(100 * len(above) / len(sizes)) if sizes else 0,
|
||||
"tokens": round(100 * sum(s - limit for s in above) / total) if total else 0,
|
||||
}
|
||||
out.append(row)
|
||||
return out
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
|
||||
parser.add_argument("--root", required=True)
|
||||
parser.add_argument("--days", type=int, required=True)
|
||||
parser.add_argument("--threshold", type=int, action="append", required=True)
|
||||
parser.add_argument("--json", action="store_true")
|
||||
args = parser.parse_args()
|
||||
root = Path(args.root).expanduser()
|
||||
if not root.is_dir() or args.days < 1:
|
||||
fail("--root must be a directory and --days at least 1")
|
||||
report = rows(root, args.days, sorted(args.threshold), time.time())
|
||||
if args.json:
|
||||
print(json.dumps(report, indent=1))
|
||||
return 0
|
||||
for row in report:
|
||||
over = " ".join(f">{k}: {v['turns']}% {v['tokens']}%" for k, v in row["over"].items())
|
||||
print(f"{row['day']}\t{row['turns']}\t{row['tokens']}\t{row['median']}\t{row['p90']}\t{row['compactions']}\t{over}")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,64 @@
|
||||
"""Exercise context.py: one count per message id, days, compactions and the threshold shares."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import datetime
|
||||
import json
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parents[1]
|
||||
SCRIPT = REPO_ROOT / "skills" / "kvb-skill-silence" / "scripts" / "context.py"
|
||||
|
||||
|
||||
def turn(day: str, mid: str, cached: int) -> str:
|
||||
usage = {"input_tokens": 100, "cache_read_input_tokens": cached, "cache_creation_input_tokens": 0}
|
||||
return json.dumps({"timestamp": f"{day}T10:00:00Z", "message": {"id": mid, "usage": usage}})
|
||||
|
||||
|
||||
class TestContext(unittest.TestCase):
|
||||
def setUp(self):
|
||||
tmp = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(tmp.cleanup)
|
||||
self.root = Path(tmp.name)
|
||||
today = datetime.datetime.now(datetime.UTC).date()
|
||||
self.today, self.yesterday = today.isoformat(), (today - datetime.timedelta(days=1)).isoformat()
|
||||
lines = [
|
||||
turn(self.today, "m1", 400_000),
|
||||
turn(self.today, "m1", 400_000),
|
||||
turn(self.today, "m2", 49_900),
|
||||
turn(self.yesterday, "m3", 599_900),
|
||||
json.dumps({"timestamp": f"{self.today}T11:00:00Z", "type": "system", "subtype": "compact_boundary"}),
|
||||
"not json",
|
||||
]
|
||||
(self.root / "project").mkdir()
|
||||
(self.root / "project" / "session.jsonl").write_text("\n".join(lines) + "\n", encoding="utf-8")
|
||||
|
||||
def _run(self, *args: str) -> subprocess.CompletedProcess:
|
||||
return subprocess.run(
|
||||
[sys.executable, str(SCRIPT), "--root", str(self.root), *args], capture_output=True, text=True, check=False
|
||||
)
|
||||
|
||||
def test_days_turns_compactions_and_shares(self):
|
||||
result = self._run("--days", "2", "--threshold", "300000", "--json")
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
yesterday, today = json.loads(result.stdout)
|
||||
self.assertEqual(
|
||||
(yesterday["day"], yesterday["turns"], yesterday["compactions"], yesterday["over"]["300000"]),
|
||||
(self.yesterday, 1, 0, {"turns": 100, "tokens": 50}),
|
||||
)
|
||||
self.assertEqual(
|
||||
(today["day"], today["turns"], today["tokens"], today["compactions"], today["over"]["300000"]),
|
||||
(self.today, 2, 450_100, 1, {"turns": 50, "tokens": 22}),
|
||||
)
|
||||
|
||||
def test_usage_errors_exit_2(self):
|
||||
self.assertEqual(self._run("--days", "0", "--threshold", "1").returncode, 2)
|
||||
self.assertEqual(self._run("--days", "1").returncode, 2)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -36,6 +36,7 @@ class TestDreamSkill(unittest.TestCase):
|
||||
"kvb-skill-tune",
|
||||
"kvb-skill-compile",
|
||||
"kvb-skill-essence",
|
||||
"kvb-skill-silence",
|
||||
"kvb-code-refactor",
|
||||
"kvb-code-tune",
|
||||
"kvb-agent-optimize",
|
||||
@@ -161,6 +162,10 @@ class TestSettingsRegistration(unittest.TestCase):
|
||||
self.assertEqual(gate.count("kvb-org-perform/scripts/log.py"), 1)
|
||||
self.assertEqual(post.count("kvb-org-perform/scripts/log.py"), 1)
|
||||
|
||||
def test_compact_window_is_set_once_and_never_overwritten(self):
|
||||
self.assertEqual(self._patch()["env"]["CLAUDE_CODE_AUTO_COMPACT_WINDOW"], "500000")
|
||||
self.settings.write_text(json.dumps({"env": {"CLAUDE_CODE_AUTO_COMPACT_WINDOW": "300000"}}), encoding="utf-8")
|
||||
self.assertEqual(self._patch()["env"]["CLAUDE_CODE_AUTO_COMPACT_WINDOW"], "300000")
|
||||
|
||||
def test_plugin_skills_are_not_copied_twice(self):
|
||||
skills = Path(self.tmp.name) / "skills"
|
||||
|
||||
Reference in new issue
Block a user