feat(org): kvb-org-radio and kvb-org-queue replace chair and peer messages

kvb-org-radio is the default channel to other agents: broadcast and direct reach the receiver on its next prompt, security and pan-pan mid-turn through a PostToolBatch hook, and only mayday or direct --wake wake a session, through the receiver's own asyncRewake Stop hook (a SendMessage fallback is printed for sessions without a listener). It keeps the news.jsonl store and lock, so news.py stays a forwarding shim. A mayday holds radio silence for two hours: watch.py blocks new heavy commands and the operator gets a desktop notice. hotfix.py announces merged hotfixes as security items.

kvb-org-queue grants heavy jobs by rule instead of chair messages: queue.py hold waits under a flock with disk reservation, hotfix-first and register rank order, slot limits, stage and silence checks, then runs the command and releases. Every job is recorded; history.py learns disk need per command and slot limits per kind (p90, five runs), watch.py mirrors the history to ~/.local/state. enable-plugins.js registers the new hooks behind a file check so a missing script never blocks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Kevin Veen-BirkenbachandClaude Opus 5.5 committed 2026-11-04 17:07:24 +01:00
1 parent ed5bb074c5
commit 27559dabc4
18 files changed
+2607 -225

No files matched your search

+5 -1
View File
@@ -78,6 +78,8 @@ flowchart LR
org --> org_evening[kvb-org-evening]
org --> org_sunset[kvb-org-sunset]
org --> org_sunrise[kvb-org-sunrise]
org --> org_radio[kvb-org-radio]
org --> org_queue[kvb-org-queue]
agent --> agent_optimize[kvb-agent-optimize]
agent --> agent_relax[kvb-agent-relax]
agent --> agent_sleep[kvb-agent-sleep]
@@ -175,6 +177,8 @@ flowchart LR
evening[kvb-org-evening]
sunset[kvb-org-sunset]
sunrise[kvb-org-sunrise]
radio[kvb-org-radio]
queue[kvb-org-queue]
end
subgraph cat_agent [agent]
agent_optimize[kvb-agent-optimize]
@@ -363,7 +367,7 @@ Copy the skills into a specific project (`<repo>/.agents/skills` and `<repo>/.cl
make project TARGET=/path/to/repo
```
Both commands also patch the target's `.claude/settings.local.json` — the machine-local, untracked file, so a project's reviewed `.claude/settings.json` is never rewritten by an install: the caveman and ponytail plugins are enabled so their modes auto-activate on session start, and two reminder hooks are registered — `kvb-dream` points each agent at `/kvb-dream` at most once every twelve hours (stamped per session in `~/.claude/.dream-stamps/`), and `kvb-coach` runs its short form at the first prompt of every full hour, once across all agents (stamped in `~/.claude/.coach-hours/`); the global install also enables the `kvb-dream-notify.timer` systemd user timer, which recommends `/kvb-dream` in a GNOME notification daily at 17:00 local time and catches up once after a missed run. A PreToolUse gate from the `kvb-org-organize` skill blocks edits to files in a git worktree until the session holds a claim (`skills/kvb-org-organize/scripts/council.py`), and its resource watch (`skills/kvb-org-organize/scripts/watch.py`) measures disk, Docker and caches, cleans up low-risk data and blocks new heavy jobs before the disk runs out; it stays inactive until `~/.config/agents-council/config.json` exists. Two hooks of the `kvb-org-perform` skill (`skills/kvb-org-perform/scripts/log.py`) log every Agent spawn with its task class, model and effort, and store the quality feedback a sandboxed session prints, in `~/.local/state/agents-council/subagent-model.jsonl`. Existing settings are preserved.
Both commands also patch the target's `.claude/settings.local.json` — the machine-local, untracked file, so a project's reviewed `.claude/settings.json` is never rewritten by an install: the caveman and ponytail plugins are enabled so their modes auto-activate on session start, and two reminder hooks are registered — `kvb-dream` points each agent at `/kvb-dream` at most once every twelve hours (stamped per session in `~/.claude/.dream-stamps/`), and `kvb-coach` runs its short form at the first prompt of every full hour, once across all agents (stamped in `~/.claude/.coach-hours/`); the global install also enables the `kvb-dream-notify.timer` systemd user timer, which recommends `/kvb-dream` in a GNOME notification daily at 17:00 local time and catches up once after a missed run. A PreToolUse gate from the `kvb-org-organize` skill blocks edits to files in a git worktree until the session holds a claim (`skills/kvb-org-organize/scripts/council.py`), and its resource watch (`skills/kvb-org-organize/scripts/watch.py`) measures disk, Docker and caches, cleans up low-risk data and blocks new heavy jobs before the disk runs out; it stays inactive until `~/.config/agents-council/config.json` exists. The `kvb-org-radio` skill registers three hooks: its prompt hook delivers `broadcast` and `direct` items on the next prompt, a PostToolBatch hook delivers `security`, `pan-pan` and `mayday` items mid-turn, and an `asyncRewake` Stop hook wakes an idle session for a `mayday` or a `direct` item sent with `--wake`. The PreToolUse hook of the `kvb-org-queue` skill sends heavy commands through `queue.py hold` once the config sets `queue_enforce`. Two hooks of the `kvb-org-perform` skill (`skills/kvb-org-perform/scripts/log.py`) log every Agent spawn with its task class, model and effort, and store the quality feedback a sandboxed session prints, in `~/.local/state/agents-council/subagent-model.jsonl`. Existing settings are preserved.
Restart your agent afterwards so it loads the new skills.
+16 -10
View File
@@ -33,6 +33,7 @@ const RENAMED = {
"democracy/scripts/council.py": "kvb-org-organize/scripts/council.py",
"democracy/scripts/watch.py": "kvb-org-organize/scripts/watch.py",
"subagent-model/scripts/log.py": "kvb-org-perform/scripts/log.py",
"kvb-org-organize/scripts/news.py": "kvb-org-radio/scripts/radio.py",
};
let data = {};
@@ -109,23 +110,28 @@ if (!JSON.stringify(data.hooks.PreToolUse).includes("kvb-org-organize/scripts/co
}
const SCRIPT_HOOKS = [
["kvb-org-organize/scripts/watch.py", "UserPromptSubmit", null, "watch"],
["kvb-org-organize/scripts/news.py", "UserPromptSubmit", null, "hook"],
["kvb-org-organize/scripts/watch.py", "PostToolUse", "Bash", "watch"],
["kvb-org-organize/scripts/watch.py", "PreToolUse", "Bash", "block"],
["kvb-org-perform/scripts/log.py", "PreToolUse", "Agent|Task", "spawn"],
["kvb-org-perform/scripts/log.py", "PostToolUse", "Bash", "persist"],
["kvb-org-organize/scripts/watch.py", "UserPromptSubmit", null, "watch", {}],
["kvb-org-radio/scripts/radio.py", "UserPromptSubmit", null, "hook", {}],
["kvb-org-radio/scripts/radio.py", "PostToolBatch", null, "tool", {}],
["kvb-org-radio/scripts/radio.py", "Stop", null, "listen", { asyncRewake: true, timeout: 21600 }],
["kvb-org-organize/scripts/watch.py", "PostToolUse", "Bash", "watch", {}],
["kvb-org-organize/scripts/watch.py", "PreToolUse", "Bash", "block", {}],
["kvb-org-queue/scripts/queue.py", "PreToolUse", "Bash", "block", {}],
["kvb-org-perform/scripts/log.py", "PreToolUse", "Agent|Task", "spawn", {}],
["kvb-org-perform/scripts/log.py", "PostToolUse", "Bash", "persist", {}],
];
for (const [script, event, matcher, sub] of SCRIPT_HOOKS) {
for (const [script, event, matcher, sub, extra] of SCRIPT_HOOKS) {
data.hooks[event] = data.hooks[event] || [];
const registered = data.hooks[event].some((entry) =>
(entry.hooks || []).some((hook) =>
String(hook.command).includes(script) && String(hook.command).endsWith(` ${sub}`)));
if (registered) continue;
const file = path.join(path.dirname(settingsPath), "skills", ...script.split("/"));
const entry = { hooks: [{ type: "command", command: `python3 ${JSON.stringify(file)} ${sub}` }] };
const file = JSON.stringify(path.join(path.dirname(settingsPath), "skills", ...script.split("/")));
// Trip-wire: python3 on a missing file exits 2, which blocks every prompt or tool call; the test makes it exit 1.
const command = `[ -f ${file} ] && python3 ${file} ${sub}`;
const entry = { hooks: [{ type: "command", command, ...extra }] };
data.hooks[event].push(matcher ? { matcher, ...entry } : entry);
}
fs.writeFileSync(settingsPath, JSON.stringify(data, null, 2) + "\n");
console.log(`skills: enabled the caveman and ponytail plugins (plugin skill copies dropped from this .claude), the ${REMINDERS.join(" + ")} reminders, the kvb-org-organize claim gate, news hook and resource watch, and the kvb-org-perform log in ${settingsPath}`);
console.log(`skills: enabled the caveman and ponytail plugins (plugin skill copies dropped from this .claude), the ${REMINDERS.join(" + ")} reminders, the kvb-org-organize claim gate and resource watch, the kvb-org-radio hooks, the kvb-org-queue gate, and the kvb-org-perform log in ${settingsPath}`);
+6 -7
View File
@@ -7,7 +7,8 @@ description: >
`${CLAUDE_SKILL_DIR}/scripts/hotfix.py` keeps each hotfix as a task
`hotfix/<slug>` of the `kvb-org-rasci` register, the author as R and A;
never a message or news post.
recording it merged radios a `security` item (`kvb-org-radio`), never post it
yourself.
```bash
hotfix.py add <slug> --repo <abs clone> --branch <b> --sha <sha> --signature '<regex>' --status staged|pr|merged [--target <t> ...] [--pr <n>]
@@ -39,12 +40,12 @@ hotfix.py void --repo <worktree> --target <t> [--target ...]
names the blocker. Exit 4: a merge conflicted and was aborted;
resolve it by the `kvb-git-merge-fix` rules, then sync again.
2. `pending`: exit 0 means every merged hotfix is in HEAD. `open, not
merged` is reported, never merged: name it in your report and go on.
merged` is reported, never merged: report it.
`not on origin/<default>` or exit 2: tell its A to correct the row.
## On any failure
Before triage, investigation or asking a peer: `match`, the line quoted
Before triage or asking a peer: `match`, the line quoted
literally; `foreign` marks another repository's hotfix. A merged match: sync
and run again. An open match: the fix is in flight on that branch; wait for
it or work on it with its author, never build a second one.
@@ -59,10 +60,8 @@ capacity rules).
target; on exit 3 finished targets keep their results.
- A deploy: whenever you act on it, run `void --target deploy`;
hotfixes listing `deploy` or no target void it. Only on exit 3 stop it at
the next safe point, between stages and never inside a task. Never on a
timer.
- After exit 3: post the release to the news, sync, and request the chair's
grant again before rerunning.
the next safe point between stages. Never on a timer.
- After exit 3: sync, then rerun through `queue.py hold` (`kvb-org-queue`).
## Rules
+39 -5
View File
@@ -22,11 +22,13 @@ Args:
--target is a make target or `deploy` the fix turns green, none listed
means every target; --pr is the pull request number.
update <slug> --status staged|pr|merged [--sha <commit>] [--pr <n>]
[--repo <abs clone>] [--default <branch>]: move a hotfix on. --sha
resolves in the recorded git common dir or in --repo, which must be
the same repository unless the recorded one is unknown, and records
it anew. --status merged needs --sha, the commit as it sits on
origin/<default> after a fetch.
[--repo <abs clone>] [--default <branch>] [--session <id>]: move a
hotfix on. --sha resolves in the recorded git common dir or in
--repo, which must be the same repository unless the recorded one is
unknown, and records it anew. --status merged needs --sha, the commit
as it sits on origin/<default> after a fetch. add and update with
--status merged post a kvb-org-radio security item; a failed post
only warns.
pending --repo <dir> [--target <t> ...]: every hotfix of that repository
against HEAD; with --target only those that turn one of the targets
green, as void counts them.
@@ -127,6 +129,7 @@ SKILLS = Path(__file__).resolve().parents[2]
RASCI = SKILLS / "kvb-org-rasci" / "scripts" / "rasci.py"
GATE = SKILLS / "kvb-check-gate" / "scripts" / "gate.py"
WATCH = SKILLS / "kvb-org-organize" / "scripts" / "watch.py"
RADIO = SKILLS / "kvb-org-radio" / "scripts" / "radio.py"
PREFIX = "hotfix/"
STAGES = ("staged", "pr", "merged")
REGISTER_STATUS = {"staged": "running", "pr": "running", "merged": "done"}
@@ -584,9 +587,37 @@ def cmd_add(rasci: ModuleType, args: argparse.Namespace) -> int:
fail(f"{task} is already in the register; move it on with update")
tasks[task] = entry
print(f"added {task}")
if args.status == "merged":
announce(task, entry, args.session)
return 0
def announce(task: str, entry: dict, session: str | None) -> None:
"""Post a merged hotfix as a kvb-org-radio security item; never fails the caller.
Args:
task: the register task, hotfix/<slug>.
entry: its register row.
session: the caller's --session, for runtimes without CLAUDE_CODE_SESSION_ID.
"""
if not RADIO.is_file():
return
text = (
f"HOTFIX merged: {task} {entry.get('sha', '-')[:12]} on {repository(entry)}; "
"hotfix.py sync before your next deploy, merge or gate"
)
command = [sys.executable, str(RADIO), "send", "security", text]
if session:
command += ["--session", session]
try:
result = subprocess.run(command, capture_output=True, text=True, timeout=30, check=False)
except (OSError, subprocess.TimeoutExpired) as error:
print(f"hotfix: radio announcement failed: {error}", file=sys.stderr)
return
if result.returncode != 0:
print(f"hotfix: radio announcement failed: {result.stderr.strip()}", file=sys.stderr)
def located(task: str, entry: dict, given: str | None) -> tuple[str, dict]:
if given is None:
common = entry.get("common_dir") or ""
@@ -624,6 +655,8 @@ def cmd_update(rasci: ModuleType, args: argparse.Namespace) -> int:
entry["stage"] = args.status
rasci.apply_status(entry, REGISTER_STATUS[args.status])
print(f"updated {task}")
if args.status == "merged":
announce(task, entry, args.session)
return 0
@@ -856,6 +889,7 @@ def main() -> int:
p.add_argument("--pr", type=int)
p.add_argument("--repo")
p.add_argument("--default")
p.add_argument("--session")
p.set_defaults(func=cmd_update)
p = sub.add_parser("pending")
p.add_argument("--repo", required=True)
+9 -10
View File
@@ -27,7 +27,7 @@ outranks them and appoints or recalls at will.
4. Elected: complete the task the claim entered in the `kvb-org-rasci`
register (objective, description, priority, roles); the listed claim
announces it.
5. Not elected: post your findings to the holder via the news and stop.
5. Not elected: radio your findings to the holder (`direct`) and stop.
A hook (`council.py gate`) blocks worktree edits before a claim. Between
machines the earliest announcement by message time wins; each host's claim
@@ -41,9 +41,9 @@ ends or goes stale, or after a restart.
on `grant`.
- `make test`, `make test-lint`, quality gates, deploys, iteration loops,
swarm, image builds, cloud, shared-data cleanup, anything with Docker or
over a few minutes: ask the chair (what, where, how long, disk), wait for
the grant, post the release to the news. In a worktree they run through
`gate.py run` or `gate.py hold` (`kvb-check-gate`).
over a few minutes: run through `queue.py hold` (`kvb-org-queue`), which
grants by rule and releases on exit; the chair only handles exceptions. In
a worktree wrap `gate.py run` or `gate.py hold` (`kvb-check-gate`).
- No running job is cut short for capacity; only the operator orders one
down. Exception: void by a missing merged hotfix (`kvb-git-hotfix`).
`RESOURCE` hook lines address the chair; others adjust plans.
@@ -55,13 +55,12 @@ ends or goes stale, or after a restart.
Read `peers.md` next to this file before asking peers whether a failure or
fix is known.
- Never poll: `council.py list`, the register and the news answer first.
- Never poll: `council.py list`, the register and the radio answer first.
- Hotfixes: `${CLAUDE_SKILL_DIR}/../kvb-git-hotfix/scripts/hotfix.py add` and
`update`, never a message or news post.
- Status that asks nothing (a failure, a freed slot, a result):
`${CLAUDE_SKILL_DIR}/scripts/news.py post "<text>"` (`--to <name>` for one
peer).
- `SendMessage` only when the recipient must act or answer: one batched
`update`; merged ones go out as radio `security` by themselves.
- Agents are reached through `kvb-org-radio`; status that asks nothing as
`broadcast` or `direct`.
- `direct --wake` only when the recipient must act or answer: one batched
message per peer per event, terse, no greeting, at most five lines unless
evidence needs more, nothing already acknowledged. A message asking
nothing gets no reply; end that turn in one line.
+13 -11
View File
@@ -5,17 +5,19 @@
- An operator-appointed allocator holds the chair: its claim says
`appointed: operator`, and no election replaces it while its session runs and
the operator has not ended the appointment.
- The chair keeps one queue: a hotfix gate (a branch from the default branch
that fixes it) takes the next free gate or test slot before every other
line, the rest follows the WSJF rank of the `kvb-org-wsjf` skill (it asks its
subordinates for the Fibonacci estimates); a running gate is never cut short
for a hotfix's slot, only when the hotfix voids it (`kvb-git-hotfix`). It
answers every request with its queue position through
`news.py post --to <name>` and sends the grant itself as a message (the
requester acts on it), enforces the limits the operator set, and checks
every grant with
`watch.py admit --need-gb <N>` (or `--kind <slot>`; test kinds also need CPU
and RAM below the configured limits).
- The queue of the `kvb-org-queue` skill grants heavy jobs by rule, with no
request, position post or grant message: a hotfix gate (a branch from the
default branch that fixes it, checked against its `hotfix/` register row)
before every other line, the rest by the register rank, that is the
operator's pins, then the WSJF of the `kvb-org-wsjf` skill; slot limits
from `slot_limits` in the config, disk reserved per granted job, CPU and RAM
through `watch.py`, nothing new at the stop stage or during a mayday. A
running gate is never cut short for a hotfix's slot, only when the hotfix
voids it (`kvb-git-hotfix`).
- The chair handles what the rule cannot: it collects the Fibonacci
estimates (an unestimated task ranks last), places a job the rule leaves
waiting too long, and carries out the operator's overrides. `queue.py list`
shows the queue.
- A running job (deploy, gate, test, swarm run, build) is never cut short to
free capacity. The chair neither asks nor orders its owner to stop, take
down or kill it, not for an operator-prioritised slot and not for a
+12 -142
View File
@@ -1,161 +1,31 @@
#!/usr/bin/env python3
"""Share status news between agents without waking them: each session sees it with its next turn.
Environment:
AGENTS_COUNCIL_ROOT: council directory; the news is <root>/news.jsonl and
each session's read mark <root>/news-seen/<session id>.
/tmp/agents-council unless set.
CLAUDE_CODE_SESSION_ID: the calling session.
"""Forward the old news.py calls to radio.py of the kvb-org-radio skill, which owns the same store.
Args:
post "<text>" [--to <name> ...] [--session <id>]: add one item, for every
session or only the named ones.
hook: UserPromptSubmit hook; reads the hook JSON on stdin, prints the items
of the last 24 hours this session has not seen (at most 10, own posts
left out) and marks them seen. Never fails the prompt.
show [--all] [--session <id>]: print the unseen items, or every kept one,
without marking them.
post "<text>" [--to <name> ...] [--session <id>], hook, show [--all]
[--session <id>]: passed to radio.py unchanged; see its docstring.
Exit codes:
0 done, 2 usage error.
radio.py's; when radio.py is missing, 0 for hook and 2 for the rest.
"""
from __future__ import annotations
import argparse
import datetime
import fcntl
import importlib.util
import json
import os
import sys
import tempfile
import time
from pathlib import Path
from types import ModuleType
ROOT = Path(os.environ.get("AGENTS_COUNCIL_ROOT", "/tmp/agents-council"))
NEWS = ROOT / "news.jsonl"
SEEN = ROOT / "news-seen"
LOCK = ROOT / "news.lock"
COUNCIL = Path(__file__).resolve().with_name("council.py")
WINDOW = 86400
SHOWN = 10
KEPT = 500
def fail(message: str) -> None:
print(f"news: {message}", file=sys.stderr)
sys.exit(2)
def council() -> ModuleType:
spec = importlib.util.spec_from_file_location("council", COUNCIL)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
def items() -> list[dict]:
try:
lines = NEWS.read_text(encoding="utf-8").splitlines()
except OSError:
return []
rows = []
for line in lines:
try:
row = json.loads(line)
except ValueError:
continue
if isinstance(row, dict) and isinstance(row.get("ts"), int | float):
rows.append(row)
return rows
def unseen(sid: str, name: str | None) -> list[dict]:
try:
mark = float((SEEN / sid).read_text(encoding="utf-8"))
except (OSError, ValueError):
mark = 0.0
floor = max(mark, time.time() - WINDOW)
return [
row
for row in items()
if row["ts"] > floor
and row.get("from") != name
and (not row.get("to") or name in row["to"])
]
def line(row: dict) -> str:
stamp = datetime.datetime.fromtimestamp(row["ts"]).strftime("%H:%M")
target = f" to {', '.join(row['to'])}" if row.get("to") else ""
return f"NEWS {stamp} {row.get('from', '?')}{target}: {row.get('text', '')}"
def cmd_post(args: argparse.Namespace, cn: ModuleType) -> int:
sid, _ = cn.caller(args.session)
text = " ".join(args.text.split())
if not text:
fail("post needs a text")
entry = {"ts": time.time(), "from": cn.live_sessions().get(sid) or sid, "to": args.to or [], "text": text}
ROOT.mkdir(parents=True, exist_ok=True)
with LOCK.open("w") as lock:
fcntl.flock(lock, fcntl.LOCK_EX)
rows = [*items()[-(KEPT - 1):], entry]
with tempfile.NamedTemporaryFile("w", dir=ROOT, delete=False, encoding="utf-8") as handle:
handle.write("".join(json.dumps(row) + "\n" for row in rows))
os.replace(handle.name, NEWS)
print(f"posted to {', '.join(entry['to']) or 'all'}")
return 0
def cmd_hook(_: argparse.Namespace, cn: ModuleType) -> int:
try:
payload = json.load(sys.stdin)
sid = payload.get("session_id") or os.environ.get("CLAUDE_CODE_SESSION_ID")
if not sid:
return 0
rows = unseen(sid, cn.live_sessions().get(sid))
if not rows:
return 0
for row in rows[-SHOWN:]:
print(line(row))
if len(rows) > SHOWN:
print(f"NEWS {len(rows) - SHOWN} older items: news.py show")
SEEN.mkdir(parents=True, exist_ok=True)
(SEEN / sid).write_text(str(rows[-1]["ts"]), encoding="utf-8")
except (OSError, ValueError, AttributeError):
return 0
return 0
def cmd_show(args: argparse.Namespace, cn: ModuleType) -> int:
if args.all:
rows = items()
else:
sid, _ = cn.caller(args.session)
rows = unseen(sid, cn.live_sessions().get(sid))
for row in rows:
print(line(row))
return 0
RADIO = Path(__file__).resolve().parents[2] / "kvb-org-radio" / "scripts" / "radio.py"
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
sub = parser.add_subparsers(dest="command", required=True)
p = sub.add_parser("post")
p.add_argument("text")
p.add_argument("--to", action="append")
p.add_argument("--session")
p.set_defaults(func=cmd_post)
sub.add_parser("hook").set_defaults(func=cmd_hook)
p = sub.add_parser("show")
p.add_argument("--all", action="store_true")
p.add_argument("--session")
p.set_defaults(func=cmd_show)
args = parser.parse_args()
return args.func(args, council())
if not RADIO.is_file():
# A UserPromptSubmit hook exiting 2 blocks every prompt of the session.
if sys.argv[1:2] == ["hook"]:
return 0
print(f"news: {RADIO} is missing; install the kvb-org-radio skill", file=sys.stderr)
return 2
os.execv(sys.executable, [sys.executable, str(RADIO), *sys.argv[1:]])
if __name__ == "__main__":
+59 -5
View File
@@ -3,9 +3,13 @@
Usage:
watch.py watch hook (UserPromptSubmit, PostToolUse): measure at most once
per interval machine-wide, act on the stage, show alerts
per interval machine-wide, act on the stage, show alerts,
and append the kvb-org-queue history rows of
<root>/queue-history.jsonl whose id it lacks to
~/.local/state/agents-council/queue-history.jsonl
watch.py block hook (PreToolUse on Bash): exit 2 for a heavy job while the
stage is stop or critical
stage is stop or critical, or while a kvb-org-radio mayday
holds radio silence (<root>/radio-silence.json, until > now)
watch.py admit (--need-gb N | --kind KIND)
exit 0 when the grant leaves at least the stop threshold
free, 3 otherwise
@@ -35,6 +39,7 @@ kind listed in kinds also needs CPU busy and RAM used below the limits.
from __future__ import annotations
import argparse
import fcntl
import json
import os
import re
@@ -53,7 +58,10 @@ CONFIG = Path(
)
STAGES = ("ok", "warn", "stop", "critical")
STATE = ROOT / "watch.json"
SILENCE = ROOT / "radio-silence.json"
ALERTS = ROOT / "alerts.jsonl"
HISTORY = ROOT / "queue-history.jsonl"
KEPT_HISTORY = Path.home() / ".local" / "state" / "agents-council" / "queue-history.jsonl"
ALERT_TTL = 3600
LOCK_TTL = 1800
GB = 1e9
@@ -180,8 +188,43 @@ def act(cfg: dict, stage: str, previous: str, free: float) -> None:
notify(cfg, f"Agents council: {head}. Heavy jobs blocked; a running stack needs the operator's OK to go.")
def history_rows(text: str) -> list[tuple[str, str]]:
rows = []
for line in text.splitlines():
try:
row = json.loads(line)
except ValueError:
continue
if isinstance(row, dict) and "id" in row:
rows.append((row["id"], line))
return rows
def keep_history() -> None:
# The agents' sandbox cannot write ~/.local/state and /tmp is wiped on reboot, so this hook keeps the rows.
# The kept file takes the source's mtime: an unchanged source is skipped without parsing.
try:
source = HISTORY.stat().st_mtime_ns
if KEPT_HISTORY.exists() and KEPT_HISTORY.stat().st_mtime_ns == source:
return
KEPT_HISTORY.parent.mkdir(parents=True, exist_ok=True)
with KEPT_HISTORY.open("a+", encoding="utf-8") as kept:
fcntl.flock(kept, fcntl.LOCK_EX)
kept.seek(0)
known = {row_id for row_id, _ in history_rows(kept.read())}
for row_id, line in history_rows(HISTORY.read_text(encoding="utf-8")):
if row_id not in known:
known.add(row_id)
kept.write(line + "\n")
kept.flush()
os.utime(KEPT_HISTORY, ns=(source, source))
except (OSError, ValueError, TypeError):
pass
def cmd_watch(cfg: dict) -> int:
payload = json.load(sys.stdin)
keep_history()
now = time.time()
state = read_json(STATE)
if now - state.get("checked", 0) >= cfg["interval_seconds"]:
@@ -216,18 +259,29 @@ def executable_text(command: str) -> str:
return QUOTED.sub("''", HEREDOC.sub("", command))
def silence() -> dict | None:
state = read_json(SILENCE)
return state if state.get("until", 0) > time.time() else None
def cmd_block(cfg: dict) -> int:
payload = json.load(sys.stdin)
command = executable_text((payload.get("tool_input") or {}).get("command", ""))
stage = read_json(STATE).get("stage", "ok")
if stage not in ("stop", "critical"):
mayday = silence()
if stage not in ("stop", "critical") and not mayday:
return 0
if any(re.search(p, command) for p in cfg["never_block"]):
return 0
if any(re.search(p, command) for p in cfg["heavy_commands"]):
reason = (
f"a mayday by {mayday.get('by', '?')} holds radio silence: {mayday.get('text', '')}; "
"it ends with radio.py clear"
if mayday
else f"resource stage is {stage}; new heavy jobs are blocked until the disk recovers"
)
print(
f"kvb-org-organize: resource stage is {stage}; new heavy jobs are blocked until "
"the disk recovers. Teardown commands still run. Ask the resource chair.",
f"kvb-org-organize: {reason}. Teardown commands still run. Ask the resource chair.",
file=sys.stderr,
)
return 2
+91
View File
@@ -0,0 +1,91 @@
---
name: kvb-org-queue
description: >
Run every heavy job (gate, deploy, build, swarm) through a deterministic
queue instead of asking the chair: hotfix first, then register rank.
---
`${CLAUDE_SKILL_DIR}/scripts/queue.py` replaces asking the chair for heavy
slots with one queue that every waiting agent evaluates by the same rules.
## Run a heavy job
```bash
queue.py hold --kind <kind> --task <register task> -- <command>
```
Start it in the background (`run_in_background` or equivalent) and end the
turn: hold waits, runs the command on its grant and exits with its code.
- `<kind>`: a key of `slot_need_gb` in `~/.config/agents-council/config.json`.
- `<register task>`: your open `kvb-org-rasci` task; its rank places you.
- `--hotfix <slug>`: the job proves the register row `hotfix/<slug>`
(`kvb-git-hotfix`) and goes first.
- `--need-gb <N>` overrides the learned disk need.
- Start hold in the job's worktree: its git toplevel is the workdir. Other
runtimes add `--session <id>`.
- In a worktree the command stays `gate.py run|hold` (`kvb-check-gate`):
`queue.py hold ... -- python3 <gate.py> run ...`.
Light runs (no Docker, about a minute) self-admit with
`watch.py admit --kind test-nostack` (`kvb-org-organize`).
## Order and grant
Every 20 s each waiting hold evaluates the whole queue under one lock:
1. Hotfix jobs first, then the register rank (`rasci.py show`), one place
earlier per whole hour waited, then arrival.
2. In that order, a job is blocked at disk stage stop or critical, at warn
when it needs disk, during a radio-silence mayday, while its kind's slots
are taken, or while its workdir runs another job.
3. A job whose need would cut below the stop threshold reserves that disk
against smaller jobs behind it.
4. An unblocked job counts as granted for the jobs behind it: none takes its
slot.
5. Kinds in `load_limits.kinds` also need CPU and RAM below the limits.
A waiting hold prints a stderr line when its position or reason changes.
## History and learning
Every finished job appends a row to the council's `queue-history.jsonl`:
signature (`make <target>`, `<script>.py <subcommand>`), kind, minutes, exit
code, and disk, RAM and CPU at the grant and at their 20 s extremes. The
`watch` hook of `kvb-org-organize` keeps them in
`~/.local/state/agents-council/` across reboots.
From the last 30 days, once 5 runs exist:
- per signature, the disk need is p90 disk taken plus 1 GB;
- per kind, the slot limit is how many p90 RAM and CPU shares fit between the
median load at the grant and `load_limits`, at least 1.
Until then the seeds hold: `slot_need_gb[kind]` for disk, 1 slot per kind.
The optional `slot_limits {kind: N}` only caps a limit from above.
## Inspect and cancel
```bash
queue.py list
queue.py history
queue.py cancel <id>
```
`list` shows each job's position, need, wait, ETA and blocking reason; dead
holders drop out. The ETA adds the median minutes left of running jobs
and of the jobs ahead, `-` while one is unknown. `history` prints the learned
values per signature and per kind, with each limit's source. `cancel` removes
your own waiting job; its hold exits 3.
## Rules
- Never ask the chair for a heavy slot or announce a grant; the chair handles
only exceptions such as a stuck queue or a hung job (`capacity.md` of
`kvb-org-organize`).
- A running job is never cut short for the queue; only the operator orders
one down, and `kvb-git-hotfix` may void a gate.
- With `queue_enforce: true`, the `block` hook (PreToolUse on Bash) refuses
`heavy_commands` and `gate.py run|hold` outside `queue.py hold`.
- Without the config, hold exits 2: tell the operator and ask the chair
(`kvb-org-organize`) until it exists.
+288
View File
@@ -0,0 +1,288 @@
"""Record the jobs of the kvb-org-queue and learn disk need per command and slot limits per kind.
Pure functions over rows and paths: queue.py passes the council history file
and the copy the kvb-org-organize watch hook keeps in ~/.local/state.
A row: id, kind, signature, workdir, start, end (epoch seconds), minutes, exit,
free_before_gb, min_free_gb, disk_gb (free_before_gb - min_free_gb, at least
0), mem_before, mem_max, cpu_before, cpu_max (percent), concurrent (other
active jobs at the grant).
"""
from __future__ import annotations
import json
import math
import os
import re
import statistics
from collections.abc import Iterable, Iterator
from pathlib import Path
MIN_RUNS = 5
WINDOW_SECONDS = 30 * 86400
DISK_MARGIN_GB = 1
MIN_SHARE_PERCENT = 1
SEED_LIMIT = 1
INTERPRETERS = re.compile(r"python[\d.]*|bash|sh|zsh|node|perl|ruby")
MAKE_VALUE_OPTIONS = frozenset({"-C", "-f", "-I", "-o", "-W"})
SIGNATURE_HEADER = ("Signature", "Kind", "Runs", "Median min", "p90 disk GB", "p90 RAM %", "Need GB")
KIND_HEADER = ("Kind", "Runs", "p90 RAM share %", "p90 CPU share %", "Slot limit", "Source")
def first_operand(tokens: Iterable[str]) -> list[str]:
return [token for token in tokens if not token.startswith("-")][:1]
def make_target(tokens: Iterator[str]) -> list[str]:
for token in tokens:
if token in MAKE_VALUE_OPTIONS:
next(tokens, None)
# A bare number is the count of a preceding -j or -l, not a target.
elif not (token.startswith("-") or "=" in token or token.isdigit()):
return [token]
return []
def signature(argv: list[str]) -> str:
"""Name a command apart from its paths, options and variables.
Args:
argv: the command as run.
Returns:
`make <first target>` for make, skipping -C, -f, -I, -o and -W with their
values, other options, job counts and VAR=value; `<script basename>
<first operand>` for an interpreter running a script (python, bash, ...)
and `<interpreter> -c` for inline code; else `<argv[0] basename> <first
operand>`. An operand is an argument not starting with `-`; a missing
target or operand is left out.
"""
name = os.path.basename(argv[0])
tokens = iter(argv[1:])
if name == "make":
return " ".join([name, *make_target(tokens)])
if INTERPRETERS.fullmatch(name):
for token in tokens:
if token == "-c":
return f"{name} -c"
if not token.startswith("-"):
name = os.path.basename(token)
break
return " ".join([name, *first_operand(tokens)])
def row(job: dict, start: float, end: float, code: int, samples: list[dict[str, float]], concurrent: int) -> dict:
"""Build the history row of a finished job.
Args:
job: its id, kind, signature and workdir.
start: epoch seconds the command started.
end: epoch seconds it ended.
code: its exit code.
samples: {free_gb, mem, cpu} measurements, the first taken at the grant.
concurrent: other active jobs at the grant.
"""
before = samples[0]
min_free = min(s["free_gb"] for s in samples)
return {
**job,
"start": start,
"end": end,
"minutes": round((end - start) / 60, 2),
"exit": code,
"free_before_gb": round(before["free_gb"], 2),
"min_free_gb": round(min_free, 2),
"disk_gb": round(max(0.0, before["free_gb"] - min_free), 2),
"mem_before": round(before["mem"], 1),
"mem_max": round(max(s["mem"] for s in samples), 1),
"cpu_before": round(before["cpu"], 1),
"cpu_max": round(max(s["cpu"] for s in samples), 1),
"concurrent": concurrent,
}
def record(path: Path, entry: dict) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
with path.open("a", encoding="utf-8") as handle:
handle.write(json.dumps(entry, sort_keys=True) + "\n")
def read(paths: Iterable[Path]) -> list[dict]:
"""Merge the rows of every history file.
Args:
paths: JSON-lines files; a missing file holds no rows and a line that is
no JSON object with an id is skipped.
Returns:
The rows, the first one of each id winning.
"""
rows: dict[str, dict] = {}
for path in paths:
try:
lines = path.read_text(encoding="utf-8").splitlines()
except FileNotFoundError:
continue
for line in lines:
try:
entry = json.loads(line)
except ValueError:
continue
if isinstance(entry, dict) and "id" in entry:
rows.setdefault(entry["id"], entry)
return list(rows.values())
def p90(values: list[float]) -> float:
return statistics.quantiles(values, n=10, method="inclusive")[-1]
def shares(rows: list[dict], resource: str) -> list[float]:
return [r[f"{resource}_max"] - r[f"{resource}_before"] for r in rows]
def grouped(rows: list[dict], key: str) -> dict[str, list[dict]]:
groups: dict[str, list[dict]] = {}
for entry in rows:
groups.setdefault(entry[key], []).append(entry)
return groups
def command_stats(rows: list[dict]) -> dict:
stats = {
"kinds": sorted({r["kind"] for r in rows}), "runs": len(rows),
"minutes": None, "disk_gb": None, "ram_share": None, "need_gb": None,
}
if len(rows) >= MIN_RUNS:
disk = p90([r["disk_gb"] for r in rows])
stats.update(
minutes=statistics.median(r["minutes"] for r in rows), disk_gb=disk,
ram_share=p90(shares(rows, "mem")), need_gb=math.ceil(disk + DISK_MARGIN_GB),
)
return stats
def fit(ceiling: float, rows: list[dict], resource: str, share: float) -> int:
before = statistics.median(r[f"{resource}_before"] for r in rows)
return math.floor((ceiling - before) / max(share, MIN_SHARE_PERCENT))
def kind_stats(rows: list[dict], limits: dict) -> dict:
stats = {"runs": len(rows), "ram_share": None, "cpu_share": None, "limit": None}
if len(rows) >= MIN_RUNS:
ram, cpu = p90(shares(rows, "mem")), p90(shares(rows, "cpu"))
stats.update(ram_share=ram, cpu_share=cpu, limit=max(1, min(
fit(limits["mem_percent"], rows, "mem", ram), fit(limits["cpu_percent"], rows, "cpu", cpu),
)))
return stats
def learn(rows: list[dict], now: float, cfg: dict) -> tuple[dict[str, dict], dict[str, dict]]:
"""Derive disk need per command and slot limit per kind from recent rows.
Args:
rows: history rows, as read() returns them.
now: epoch seconds; rows that ended more than WINDOW_SECONDS earlier are
ignored.
cfg: the config; load_limits.mem_percent and cpu_percent bound a limit.
Returns:
(signatures, kinds). signatures: {signature: {kinds, runs, minutes,
disk_gb, ram_share, need_gb}}; kinds: {kind: {runs, ram_share,
cpu_share, limit}}. With fewer than MIN_RUNS rows every value but kinds
and runs is None. minutes is the median, disk_gb, ram_share and
cpu_share the p90 of the disk taken and of max minus before in
percentage points; need_gb is ceil(disk_gb + DISK_MARGIN_GB); limit is
max(1, min over RAM and CPU of floor((load limit - median before) /
max(share, MIN_SHARE_PERCENT))).
"""
recent = [r for r in rows if now - r["end"] <= WINDOW_SECONDS]
signatures = {name: command_stats(group) for name, group in grouped(recent, "signature").items()}
kinds = {name: kind_stats(group, cfg["load_limits"]) for name, group in grouped(recent, "kind").items()}
return signatures, kinds
def need_gb(signatures: dict[str, dict], name: str) -> int | None:
return signatures[name]["need_gb"] if name in signatures else None
def slot_limits(kinds: dict[str, dict], caps: dict[str, int], names: Iterable[str]) -> dict[str, tuple[int, str]]:
"""Decide the slot limit of every kind.
Args:
kinds: the kind map of learn().
caps: the config's optional slot_limits {kind: N}; a cap only lowers.
names: kinds to decide besides those of kinds and caps.
Returns:
{kind: (limit, source)}, sorted by kind: the learned limit (source
`learned`), else SEED_LIMIT (`seed`), lowered to a smaller cap
(`capped by config`).
"""
limits = {}
for kind in sorted({*names, *kinds, *caps}):
limit, source = SEED_LIMIT, "seed"
if kind in kinds and kinds[kind]["limit"] is not None:
limit, source = kinds[kind]["limit"], "learned"
if kind in caps and caps[kind] < limit:
limit, source = caps[kind], "capped by config"
limits[kind] = (limit, source)
return limits
def eta(signatures: dict[str, dict], running: list[tuple[str, float]], ahead: list[str], now: float) -> float | None:
"""Estimate the minutes until a waiting job starts, or until a running one ends.
Args:
signatures: the signature map of learn(); its median minutes drive the
estimate.
running: (signature, grant epoch seconds) of the running jobs counted.
ahead: signatures of the waiting jobs before it.
now: epoch seconds.
Returns:
The remaining median minutes of running plus the median minutes of
ahead, as if run one after another; None when one has no median yet.
"""
def median(name: str) -> float | None:
return signatures[name]["minutes"] if name in signatures else None
parts = [(median(name), (now - granted) / 60) for name, granted in running]
parts += [(median(name), 0.0) for name in ahead]
if any(minutes is None for minutes, _ in parts):
return None
return sum(max(0.0, minutes - elapsed) for minutes, elapsed in parts)
def cell(value: float | None) -> str:
return "-" if value is None else f"{value:.1f}"
def table(header: tuple[str, ...], rows: list[list[str]]) -> str:
lines = [header, ["---"] * len(header), *rows]
return "".join(f"| {' | '.join(line)} |\n" for line in lines)
def report(signatures: dict[str, dict], kinds: dict[str, dict], limits: dict[str, tuple[int, str]]) -> str:
"""Render the learned values as two markdown tables.
Args:
signatures: the signature map of learn().
kinds: the kind map of learn().
limits: slot_limits() for every kind to list.
Returns:
A table per signature, a blank line, a table per kind.
"""
commands = [
[name, ", ".join(s["kinds"]), str(s["runs"]), cell(s["minutes"]), cell(s["disk_gb"]),
cell(s["ram_share"]), "-" if s["need_gb"] is None else str(s["need_gb"])]
for name, s in sorted(signatures.items())
]
per_kind = []
for kind, (limit, source) in limits.items():
runs, ram, cpu = 0, None, None
if kind in kinds:
runs, ram, cpu = (kinds[kind][key] for key in ("runs", "ram_share", "cpu_share"))
per_kind.append([kind, str(runs), cell(ram), cell(cpu), str(limit), source])
return table(SIGNATURE_HEADER, commands) + "\n" + table(KIND_HEADER, per_kind)
+578
View File
@@ -0,0 +1,578 @@
#!/usr/bin/env python3
"""Queue the heavy jobs of parallel agents and grant them in a fixed order, without messages.
Usage:
queue.py hold --kind KIND --task TASK [--need-gb N] [--hotfix SLUG] [--session ID] -- CMD...
queue.py list
queue.py history
queue.py cancel ID [--session ID]
queue.py block hook (PreToolUse on Bash)
Environment:
AGENTS_COUNCIL_ROOT: shared state directory; /tmp/agents-council unless set,
because every agent on the host must agree on it. The queue is
<root>/queue.json, every read-modify-write holds <root>/queue.lock.
Every finished job appends a row (history.py) to
<root>/queue-history.jsonl; the watch hook of watch.py keeps a copy in
~/.local/state/agents-council/queue-history.jsonl, and reading merges
both by row id.
AGENTS_COUNCIL_CONFIG: the config of the kvb-org-organize watch.py;
~/.config/agents-council/config.json unless set.
CLAUDE_CODE_SESSION_ID: the caller's session, taken over --session; other
runtimes pass --session.
Args:
hold: enqueue CMD, then every POLL_SECONDS evaluate the whole queue under
the lock until this entry is granted; measure disk, RAM and CPU, run
CMD in the current directory with inherited stdio in a session of its
own, forward SIGINT and SIGTERM to its process group, measure every
POLL_SECONDS and at its end, record its history row and leave the
queue.
--kind: a key of slot_need_gb in the config.
--task: an open task of the kvb-org-rasci register; its rank orders
the queue.
--need-gb: disk the job takes, in GB; when omitted the need learned
for the command's signature, else slot_need_gb[kind].
--hotfix: the slug of the register row hotfix/<slug> the job proves;
hotfix jobs go first.
--session: the caller's session for runtimes other than Claude Code.
list: a header, then one line per entry: position (run for active jobs),
id, status, kind, task, need GB, minutes waited, ETA minutes (until a
running job ends or a waiting one starts, from learned medians; - while
one is unknown), workdir, blocking reason. Entries of dead holders are
dropped first.
history: a markdown table per command signature and one per kind of what
the last 30 days of history teach, with each slot limit's source.
cancel ID: remove the caller's waiting entry ID; its hold exits 3.
block: read the hook payload on stdin; with queue_enforce true in the
config, refuse a command matching heavy_commands or `gate.py run|hold`
unless it matches never_block or runs `queue.py hold`.
Config keys: slot_limits {kind: N}, optional, caps the learned slot limit of a
kind from above (history.slot_limits); queue_enforce (bool), optional, read by
block only. The watch.py keys disk, thresholds_gb (stop is the floor every
grant leaves free), slot_need_gb (the disk seed per kind), load_limits (kinds
in its kinds also need CPU and RAM below the limits; mem_percent and
cpu_percent bound the learned slot limits), heavy_commands and never_block are
used as watch.py documents them.
Exit codes:
hold: the command's exit code unchanged, 128+N when signal N ended it;
before the command started 2 on a usage or config error or when it
cannot start, 3 when cancel removed the entry, 128+N when signal N
reached hold.
list, history: 0 done, 2 config error.
cancel: 0 removed, 2 no waiting entry of the caller with that id.
block: 0 allow, 2 refuse the tool call; every error allows.
"""
from __future__ import annotations
# Trip-wire: run as a script, this file shadows the stdlib queue module, so nothing here
# may import a module that imports queue (logging.handlers, concurrent.futures, multiprocessing).
import argparse
import contextlib
import fcntl
import functools
import importlib.util
import json
import os
import re
import shlex
import signal
import subprocess
import sys
import tempfile
import time
from collections import Counter
from collections.abc import Iterator
from pathlib import Path
from types import ModuleType
from typing import NoReturn
SKILLS = Path(__file__).resolve().parents[2]
SIBLINGS = {
"watch": SKILLS / "kvb-org-organize" / "scripts" / "watch.py",
"council": SKILLS / "kvb-org-organize" / "scripts" / "council.py",
"rasci": SKILLS / "kvb-org-rasci" / "scripts" / "rasci.py",
"history": Path(__file__).resolve().parent / "history.py",
}
POLL_SECONDS = 20
TICK_SECONDS = 0.2
HOUR = 3600
CANCELLED = 3
FORWARDED = (signal.SIGINT, signal.SIGTERM)
GATE = re.compile(r"\bgate\.py\s+(run|hold)\b")
THROUGH_QUEUE = re.compile(r"\bqueue\.py\s+hold\b")
HOTFIX = "hotfix/"
@functools.cache
def sibling(name: str) -> ModuleType:
spec = importlib.util.spec_from_file_location(f"kvb_queue_{name}", SIBLINGS[name])
loaded = importlib.util.module_from_spec(spec)
spec.loader.exec_module(loaded)
return loaded
watch = sibling("watch")
history = sibling("history")
STORE = watch.ROOT / "queue.json"
LOCK = watch.ROOT / "queue.lock"
SILENCE = watch.ROOT / "radio-silence.json"
HISTORIES = (watch.HISTORY, watch.KEPT_HISTORY)
def fail(message: str) -> NoReturn:
print(f"queue: {message}", file=sys.stderr)
sys.exit(2)
def config() -> dict:
cfg = watch.load_config()
if cfg is None:
fail(f"no config at {watch.CONFIG}; the queue reads the kvb-org-organize watch config")
return cfg
def caps(cfg: dict) -> dict[str, int]:
return cfg.get("slot_limits", {})
def learned(cfg: dict, now: float) -> tuple[dict[str, dict], dict[str, dict]]:
return history.learn(history.read(HISTORIES), now, cfg)
def limits(cfg: dict, kinds: dict[str, dict], entries: list[dict]) -> dict[str, tuple[int, str]]:
return history.slot_limits(kinds, caps(cfg), {*cfg["slot_need_gb"], *(e["kind"] for e in entries)})
def signature_of(entry: dict) -> str:
return history.signature(shlex.split(entry["cmd"]))
def identity(pid: int) -> str | None:
"""Name a live process by its start time, so a reused pid does not match.
Args:
pid: the process id.
Returns:
The start time from /proc, `running` where /proc is unreadable, None for
a gone or zombie process.
"""
try:
os.kill(pid, 0)
except ProcessLookupError:
return None
except PermissionError:
pass
try:
fields = Path(f"/proc/{pid}/stat").read_text(encoding="utf-8").rsplit(")", 1)[1].split()
except OSError:
return "running"
return None if fields[0] == "Z" else fields[19]
def alive(pid: int | None, start: str | None) -> bool:
return pid is not None and start is not None and identity(pid) == start
def prune(entries: list[dict]) -> list[dict]:
return [
e for e in entries
if alive(e["pid"], e["pid_start"]) or (e["status"] == "active" and alive(e["child"], e["child_start"]))
]
def read_store() -> dict:
try:
return json.loads(STORE.read_text(encoding="utf-8"))
except FileNotFoundError:
return {"seq": 0, "entries": []}
except ValueError:
fail(f"{STORE} is not valid JSON; move it aside to start over")
@contextlib.contextmanager
def store() -> Iterator[dict]:
watch.ROOT.mkdir(parents=True, exist_ok=True)
with LOCK.open("a", encoding="utf-8") as lock:
fcntl.flock(lock, fcntl.LOCK_EX)
data = read_store()
yield data
with tempfile.NamedTemporaryFile(
"w", dir=watch.ROOT, prefix=".queue-", delete=False, encoding="utf-8"
) as tmp:
json.dump(data, tmp, indent=1, sort_keys=True)
os.replace(tmp.name, STORE)
def ordered(waiting: list[dict], ranks: dict[str, int], now: float) -> list[dict]:
def key(entry: dict) -> tuple:
hours = int((now - entry["enqueued"]) // HOUR)
rank = ranks[entry["task"]] if entry["task"] in ranks else len(ranks)
return (not entry["hotfix"], rank - hours, entry["seq"])
return sorted(waiting, key=key)
def evaluate(entries: list[dict], free_gb: float, floor: float, stage: str, silenced: bool,
slot_limits: dict[str, int], ranks: dict[str, int], now: float) -> dict[str, str | None]:
"""Decide every waiting entry in grant order, without disk or processes.
Args:
entries: the queue's entries, active and waiting.
free_gb: free disk in GB.
floor: GB every grant must leave free (thresholds_gb.stop).
stage: the watch stage: ok, warn, stop or critical.
silenced: whether a radio-silence mayday holds.
slot_limits: {kind: max concurrent jobs}.
ranks: {task: index in the register's ranking}; a task missing there
ranks after all of them.
now: epoch seconds; each whole hour waited moves an entry one rank up.
Returns:
{entry id: None when granted, else the blocking reason}, in grant order.
An unblocked entry counts as granted for the entries after it, whoever
holds it, so a lower entry never takes a higher entry's slot.
"""
active = [e for e in entries if e["status"] == "active"]
available = free_gb - sum(e["need_gb"] for e in active)
counts = Counter(e["kind"] for e in active)
busy = {e["workdir"] for e in active}
verdicts: dict[str, str | None] = {}
for entry in ordered([e for e in entries if e["status"] == "waiting"], ranks, now):
kind, need, workdir = entry["kind"], entry["need_gb"], entry["workdir"]
reason = None
if stage in ("stop", "critical"):
reason = f"disk stage {stage}"
elif stage == "warn" and need > 0:
reason = "disk stage warn admits only jobs that need no disk"
elif silenced:
reason = "radio silence"
elif kind not in slot_limits:
reason = f"slot_limits has no {kind}"
elif counts[kind] >= slot_limits[kind]:
reason = f"all {slot_limits[kind]} {kind} slots taken"
elif workdir in busy:
reason = f"{workdir} runs another job"
elif available - need < floor:
reason = f"{need:g} GB would cut into the {floor:g} GB floor; reserved"
available -= need
else:
counts[kind] += 1
busy.add(workdir)
available -= need
verdicts[entry["id"]] = reason
return verdicts
def register_ranks() -> dict[str, int]:
rasci = sibling("rasci")
return {task: index for index, task in enumerate(rasci.ranked(rasci.merged(False)))}
def stage(cfg: dict, free_gb: float, now: float) -> str:
state = watch.read_json(watch.STATE)
return state["stage"] if "stage" in state else watch.stage_for(free_gb, [], cfg, now)
def silenced(now: float) -> bool:
state = watch.read_json(SILENCE)
return "until" in state and state["until"] > now
def decide(cfg: dict, entries: list[dict], free_gb: float, now: float, kinds: dict[str, dict]) -> dict[str, str | None]:
slots = {kind: limit for kind, (limit, _) in limits(cfg, kinds, entries).items()}
return evaluate(
entries, free_gb, cfg["thresholds_gb"]["stop"], stage(cfg, free_gb, now), silenced(now),
slots, register_ranks(), now,
)
def load_reason(cfg: dict, kind: str) -> str | None:
limits = cfg["load_limits"]
if kind not in limits["kinds"]:
return None
cpu, mem = watch.load(cfg)
if cpu < limits["cpu_percent"] and mem < limits["mem_percent"]:
return None
return f"cpu or ram at the limit ({limits['cpu_percent']} %, {limits['mem_percent']} %)"
def workdir() -> str:
try:
top = subprocess.run(
["git", "rev-parse", "--show-toplevel"], capture_output=True, text=True, check=False
)
except OSError:
return os.path.realpath(os.getcwd())
path = top.stdout.strip() if top.returncode == 0 else ""
return os.path.realpath(path or os.getcwd())
def enqueue(fields: dict) -> dict:
with store() as data:
data["seq"] += 1
entry = {"id": f"{fields['kind']}-{data['seq']}", "seq": data["seq"], **fields}
data["entries"].append(entry)
return entry
def update(entry_id: str, **fields: object) -> None:
with store() as data:
for entry in data["entries"]:
if entry["id"] == entry_id:
entry.update(fields)
def leave(entry_id: str) -> None:
with store() as data:
data["entries"] = [e for e in data["entries"] if e["id"] != entry_id]
def pause(caught: list[int]) -> None:
deadline = time.monotonic() + POLL_SECONDS
while not caught and time.monotonic() < deadline:
time.sleep(TICK_SECONDS)
def wait(cfg: dict, entry_id: str, kind: str, caught: list[int]) -> tuple[int | None, int]:
"""Poll until the grant.
Args:
cfg: the config.
entry_id: this hold's entry.
kind: its kind, for the load sample.
caught: signals the caller caught; the first ends the wait.
Returns:
(None, other active jobs at the grant) once granted, else (the exit
code, 0).
"""
shown = None
while not caught:
busy_load = load_reason(cfg, kind)
free, _ = watch.disk(cfg)
now = time.time()
_, kinds = learned(cfg, now)
with store() as data:
data["entries"] = prune(data["entries"])
mine = [e for e in data["entries"] if e["id"] == entry_id]
if not mine:
print(f"queue: {entry_id} was cancelled", file=sys.stderr)
return CANCELLED, 0
verdicts = decide(cfg, data["entries"], free, now, kinds)
reason = verdicts[entry_id] or busy_load
if reason is None:
concurrent = sum(e["status"] == "active" for e in data["entries"])
mine[0].update(status="active", granted=now)
return None, concurrent
position = list(verdicts).index(entry_id) + 1
if (position, reason) != shown:
print(f"queue: {entry_id} waits at position {position}: {reason}", file=sys.stderr, flush=True)
shown = (position, reason)
pause(caught)
return 128 + caught[0], 0
def send(child: subprocess.Popen, signum: int) -> None:
if child.returncode is None:
with contextlib.suppress(ProcessLookupError, PermissionError):
os.killpg(child.pid, signum)
def measure(cfg: dict) -> dict[str, float]:
free, _ = watch.disk(cfg)
cpu, mem = watch.load(cfg)
return {"free_gb": free, "mem": mem, "cpu": cpu}
def supervise(cfg: dict, child: subprocess.Popen, samples: list[dict[str, float]]) -> int:
"""Wait for the child, measuring every POLL_SECONDS and once at its end.
Args:
cfg: the config.
child: the running command.
samples: the measurements so far; each new one is appended.
Returns:
The child's return code.
"""
code = None
while code is None:
with contextlib.suppress(subprocess.TimeoutExpired):
code = child.wait(timeout=POLL_SECONDS)
samples.append(measure(cfg))
return code
def keep(row: dict) -> None:
try:
history.record(watch.HISTORY, row)
except OSError as error:
# A full disk refuses the row; the job's exit code must still come through.
print(f"queue: no history row for {row['id']}: {error.strerror}", file=sys.stderr)
def cmd_hold(args: argparse.Namespace) -> int:
cfg = config()
if args.kind not in cfg["slot_need_gb"]:
fail(f"kind {args.kind!r} is not in slot_need_gb of {watch.CONFIG}")
rasci = sibling("rasci")
if args.task not in rasci.merged(False):
fail(f"{args.task} is no open task of the kvb-org-rasci register")
if args.hotfix is not None and HOTFIX + args.hotfix not in rasci.load():
fail(f"the register has no row {HOTFIX}{args.hotfix}")
signature = history.signature(args.command)
signatures, _ = learned(cfg, time.time())
need = args.need_gb
if need is None:
need = history.need_gb(signatures, signature)
if need is None:
need = cfg["slot_need_gb"][args.kind]
if need < 0:
fail("--need-gb must be 0 or more")
council = sibling("council")
session, _ = council.caller(args.session)
caught: list[int] = []
child: subprocess.Popen | None = None
def forward(signum: int, _frame: object) -> None:
if child is None:
caught.append(signum)
else:
send(child, signum)
for signum in FORWARDED:
signal.signal(signum, forward)
entry = enqueue({
"kind": args.kind, "task": args.task, "need_gb": need, "hotfix": args.hotfix,
"workdir": workdir(), "session": session, "name": council.live_sessions().get(session) or session,
"pid": os.getpid(), "pid_start": identity(os.getpid()), "child": None, "child_start": None,
"status": "waiting", "enqueued": time.time(), "granted": None, "cmd": shlex.join(args.command),
})
try:
code, concurrent = wait(cfg, entry["id"], args.kind, caught)
if code is not None:
return code
if caught:
return 128 + caught[0]
samples = [measure(cfg)]
start = time.time()
try:
child = subprocess.Popen(args.command, start_new_session=True)
except OSError as error:
print(f"queue: cannot start {args.command[0]}: {error.strerror}", file=sys.stderr)
return 2
for signum in caught:
send(child, signum)
update(entry["id"], child=child.pid, child_start=identity(child.pid))
code = supervise(cfg, child, samples)
code = code if code >= 0 else 128 - code
job = {"id": f"{entry['id']}-{time.time_ns()}", "kind": args.kind, "signature": signature,
"workdir": entry["workdir"]}
keep(history.row(job, start, time.time(), code, samples, concurrent))
return code
finally:
leave(entry["id"])
def cmd_list(_: argparse.Namespace) -> int:
cfg = config()
free, _ = watch.disk(cfg)
now = time.time()
signatures, kinds = learned(cfg, now)
with store() as data:
data["entries"] = prune(data["entries"])
entries = list(data["entries"])
verdicts = decide(cfg, entries, free, now, kinds)
by_id = {e["id"]: e for e in entries}
running = [(signature_of(e), e["granted"]) for e in entries if e["status"] == "active"]
rows = [
("run", e, history.eta(signatures, [(signature_of(e), e["granted"])], [], now), "-")
for e in entries if e["status"] == "active"
]
ahead: list[str] = []
for n, (entry_id, reason) in enumerate(verdicts.items(), 1):
e = by_id[entry_id]
rows.append((str(n), e, history.eta(signatures, running, ahead, now), reason or "granted on its next poll"))
ahead.append(signature_of(e))
print("position\tid\tstatus\tkind\ttask\tneed_gb\twaited_min\teta_min\tworkdir\treason")
for position, e, eta, reason in rows:
waited = ((e["granted"] or now) - e["enqueued"]) / 60
print("\t".join(
[position, e["id"], e["status"], e["kind"], e["task"], f"{e['need_gb']:g}", f"{waited:.0f}",
"-" if eta is None else f"{eta:.0f}", e["workdir"], reason]
))
return 0
def cmd_history(_: argparse.Namespace) -> int:
cfg = config()
signatures, kinds = learned(cfg, time.time())
print(history.report(signatures, kinds, limits(cfg, kinds, [])), end="")
return 0
def cmd_cancel(args: argparse.Namespace) -> int:
session, _ = sibling("council").caller(args.session)
with store() as data:
mine = [
e for e in data["entries"]
if e["id"] == args.id and e["status"] == "waiting" and e["session"] == session
]
if not mine:
fail(f"no waiting entry {args.id} of this session; `queue.py list` shows the ids")
data["entries"].remove(mine[0])
print(f"cancelled {args.id}")
return 0
def cmd_block() -> int:
payload = json.load(sys.stdin)
cfg = watch.load_config()
if cfg is None or cfg.get("queue_enforce") is not True:
return 0
command = watch.executable_text((payload.get("tool_input") or {}).get("command") or "")
if THROUGH_QUEUE.search(command) or any(re.search(p, command) for p in cfg["never_block"]):
return 0
if GATE.search(command) or any(re.search(p, command) for p in cfg["heavy_commands"]):
print(
f"kvb-org-queue: heavy jobs run through the queue: start `python3 {Path(__file__).resolve()} hold "
"--kind <kind> --task <register task> -- <command>` in the background; it runs the command on its grant.",
file=sys.stderr,
)
return 2
return 0
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
sub = parser.add_subparsers(dest="cmd", required=True)
hold = sub.add_parser("hold")
hold.add_argument("--kind", required=True)
hold.add_argument("--task", required=True)
hold.add_argument("--need-gb", type=float)
hold.add_argument("--hotfix")
hold.add_argument("--session")
hold.add_argument("command", nargs="+")
sub.add_parser("list")
sub.add_parser("history")
cancel = sub.add_parser("cancel")
cancel.add_argument("id")
cancel.add_argument("--session")
sub.add_parser("block")
args = parser.parse_args()
if args.cmd == "block":
try:
return cmd_block()
except (OSError, ValueError, KeyError, TypeError, AttributeError, re.error):
return 0
commands = {"hold": cmd_hold, "list": cmd_list, "history": cmd_history, "cancel": cmd_cancel}
return commands[args.cmd](args)
if __name__ == "__main__":
sys.exit(main())
+70
View File
@@ -0,0 +1,70 @@
---
name: kvb-org-radio
description: >
Default channel for every message to other agents: board news, direct
calls, security, pan-pan and mayday; wake a peer only when it must act.
---
`${CLAUDE_SKILL_DIR}/scripts/radio.py` is the channel every skill uses to
reach other agents, on every runtime. All classes share one store,
`news.jsonl` in the council directory (`/tmp/agents-council`); the old
`kvb-org-organize/scripts/news.py` forwards here.
```bash
radio.py send <class> "<text>" [--to <name> ...] [--wake] [--session <id>]
radio.py post "<text>" [--to <name> ...] [--session <id>]
radio.py show [--all] [--session <id>]
radio.py clear [--force] [--session <id>]
```
`post` is `send broadcast`, or `send direct` with `--to`.
## Classes
| Class | Addressee | Delivery | Wakes | Speed | Rough cost |
| --- | --- | --- | --- | --- | --- |
| broadcast | all | board line at each session's next prompt | no | next prompt | ~40 tokens per reader |
| direct | `--to` names | board line at the addressee's next prompt | only with `--wake` | next prompt; `--wake` within 15 s | ~40 tokens; a wake is a full turn of the receiver, very expensive |
| security | all | after the receiver's next tool batch, else next prompt | no | seconds while it works | ~40 tokens per reader |
| pan-pan | all | like security, listed above it | no | seconds while it works | ~40 tokens per reader |
| mayday | all | like pan-pan, plus a wake of every idle session and radio silence | yes | within 15 s | a full turn of every receiver, very expensive |
Pick the lowest class that does the job:
- **broadcast**: status that asks nothing (a result, a freed slot, a merge).
- **direct**: the same for one or a few peers.
- **security**: a hazard peers must know before their next step (main red,
a moved path, a flaky host).
- **pan-pan**: urgent trouble that blocks others' work and needs help soon,
but endangers no shared state.
- **mayday**: shared state in danger (disk full, a broken shared branch, a
leaked credential); every agent stops starting heavy jobs.
## Rules
- Wake only with `mayday` or `direct --wake`, and only when the recipient
must act or answer; this replaces `SendMessage`. For an addressee no
listener wakes yet, `send` prints `wake: SendMessage to <name>`: send that
peer the same text once with your runtime's message tool.
- Hotfixes are announced automatically when
`${CLAUDE_SKILL_DIR}/../kvb-git-hotfix/scripts/hotfix.py` records them
merged; never post them yourself.
- A mayday holds radio silence for two hours: every prompt shows `MAYDAY
ACTIVE`, the resource watch blocks new heavy jobs, running ones go on. Its
sender ends it with `radio.py clear` once resolved. Never clear another
agent's mayday; `--force` only on the operator's order.
- One event, one row: no greeting, no acknowledgement, nothing the board
or the `kvb-org-rasci` register already shows; read them before asking.
- No secret, token or credential value in a text: name where it lives.
## Delivery
On Claude Code three hooks deliver, each row once per session: `hook`
(UserPromptSubmit) prints the unseen rows, urgent first, at most 10; `tool`
(PostToolBatch) adds unseen urgent rows to a running turn; `listen` (Stop,
`asyncRewake`) wakes an idle session for a mayday or a `direct --wake`.
Subagents get nothing; their parent session reads the radio.
Other runtimes (Codex, Gemini CLI, Hermes, OpenClaw) use the same store and
scripts with `--session <id>` and read with `radio.py show` at session start
and before every heavy job; nothing wakes them.
+473
View File
@@ -0,0 +1,473 @@
#!/usr/bin/env python3
"""Reach other agents by radio: five classes, from a board line to a wake, in one shared store.
Environment:
AGENTS_COUNCIL_ROOT: council directory shared by every agent on the host;
/tmp/agents-council unless set. Rows live in <root>/news.jsonl under
<root>/news.lock, the store the old news.py reads and writes too.
AGENTS_COUNCIL_CONFIG: config file whose "notify" argv tells the operator
of a mayday; ~/.config/agents-council/config.json unless set.
CLAUDE_CODE_SESSION_ID: the calling session; hooks read the payload's
session_id first, other runtimes pass --session.
Args:
send <class> "<text>" [--to <name> ...] [--wake] [--session <id>]: add a
row. broadcast, security, pan-pan and mayday go to all and take no
--to; direct needs --to; --wake only with direct. mayday also sets
radio silence (<root>/radio-silence.json) for two hours. A wake
(mayday, direct --wake) then prints `wake: SendMessage to <name>`
for every addressee without a live listener.
post "<text>" [--to <name> ...] [--session <id>]: send broadcast, or
direct with --to.
clear [--force] [--session <id>]: end radio silence; its sender may,
anyone else only with --force.
hook: UserPromptSubmit hook; prints the active silence and the unseen
rows, mayday, pan-pan and security first, at most 10, and marks them
seen. Never fails the prompt.
tool: PostToolBatch hook; prints the unseen urgent rows as
additionalContext. Never fails.
listen: Stop hook with asyncRewake; polls every POLL_SECONDS until a
mayday or a direct --wake reaches this session, prints it to stderr
and exits 2; exits 0 once a newer listener of the session took over.
show [--all] [--session <id>]: print the active silence and the unseen
rows, or every kept row, without marking them.
Exit codes:
0 done, 2 usage error (listen: wake the session), 3 radio silence held
by another session.
"""
from __future__ import annotations
import argparse
import contextlib
import datetime
import fcntl
import functools
import importlib.util
import json
import math
import os
import subprocess
import sys
import tempfile
import time
from collections.abc import Iterator
from pathlib import Path
from types import ModuleType
# Every agent on the host must agree on one council directory and one config without being told.
ROOT = Path(os.environ.get("AGENTS_COUNCIL_ROOT", "/tmp/agents-council"))
CONFIG = Path(
os.environ.get("AGENTS_COUNCIL_CONFIG", str(Path.home() / ".config" / "agents-council" / "config.json"))
)
NEWS = ROOT / "news.jsonl"
LOCK = ROOT / "news.lock"
SEEN = ROOT / "news-seen"
STATE = ROOT / "radio-seen"
LISTEN = ROOT / "radio-listen"
SILENCE = ROOT / "radio-silence.json"
NOTIFIED = ROOT / "radio-notified"
COUNCIL = Path(__file__).resolve().parents[2] / "kvb-org-organize" / "scripts" / "council.py"
CLASSES = ("broadcast", "direct", "security", "pan-pan", "mayday")
URGENT = ("mayday", "pan-pan", "security")
WINDOW = 86400
SHOWN = 10
KEPT = 500
SILENCE_SECONDS = 7200
NOTIFY_SECONDS = 10
POLL_SECONDS = 15
NEVER = 0.0
def fail(message: str) -> None:
print(f"radio: {message}", file=sys.stderr)
sys.exit(2)
@functools.cache
def council() -> ModuleType:
spec = importlib.util.spec_from_file_location("council", COUNCIL)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
def name_of(sid: str) -> str:
return council().live_sessions().get(sid) or sid
def items() -> list[dict]:
try:
lines = NEWS.read_text(encoding="utf-8").splitlines()
except OSError:
return []
rows = []
for line in lines:
try:
row = json.loads(line)
except ValueError:
continue
if isinstance(row, dict) and isinstance(row.get("ts"), int | float):
rows.append(row)
return rows
def kind(row: dict) -> str:
if row.get("class") in CLASSES:
return row["class"]
return "direct" if row.get("to") else "broadcast"
def urgent(row: dict) -> bool:
return kind(row) in URGENT
def rank(row: dict) -> int:
return URGENT.index(kind(row)) if urgent(row) else len(URGENT)
def ordered(rows: list[dict]) -> list[dict]:
return sorted(rows, key=lambda row: (rank(row), row["ts"]))
def clock(ts: float) -> str:
return datetime.datetime.fromtimestamp(ts).strftime("%H:%M")
def line(row: dict) -> str:
target = f" to {', '.join(row['to'])}" if row.get("to") else ""
return f"RADIO {kind(row).upper()} {clock(row['ts'])} {row.get('from')}{target}: {row.get('text')}"
def for_me(row: dict, name: str) -> bool:
return row.get("from") != name and (not row.get("to") or name in row["to"])
def read_json(path: Path) -> dict:
try:
data = json.loads(path.read_text(encoding="utf-8"))
except (OSError, ValueError):
return {}
return data if isinstance(data, dict) else {}
def write_atomic(path: Path, text: str) -> None:
with tempfile.NamedTemporaryFile("w", dir=path.parent, delete=False, encoding="utf-8") as handle:
handle.write(text)
os.replace(handle.name, path)
def prompt_mark(sid: str) -> float:
try:
return float((SEEN / sid).read_text(encoding="utf-8"))
except (OSError, ValueError):
return NEVER
def mark(state: dict, key: str) -> float:
value = state.get(key)
return NEVER if value is None else value
def news_mtime() -> float | None:
try:
return NEWS.stat().st_mtime
except FileNotFoundError:
return None
@contextlib.contextmanager
def session(sid: str) -> Iterator[dict]:
STATE.mkdir(parents=True, exist_ok=True)
with (STATE / f"{sid}.lock").open("w") as lock:
fcntl.flock(lock, fcntl.LOCK_EX)
yield read_json(STATE / f"{sid}.json")
def save(sid: str, state: dict) -> None:
write_atomic(STATE / f"{sid}.json", json.dumps(state))
def hook_session() -> str | None:
payload = json.load(sys.stdin)
if payload.get("agent_id"):
return None
return payload.get("session_id") or os.environ.get("CLAUDE_CODE_SESSION_ID")
def silence() -> dict | None:
state = read_json(SILENCE)
until = state.get("until")
return state if isinstance(until, int | float) and until > time.time() else None
def silence_line(active: dict) -> str:
return (
f"MAYDAY ACTIVE since {clock(active['since'])} by {active['by']}: {active['text']}; "
"no new heavy jobs until radio.py clear"
)
def notify_once(active: dict | None) -> None:
if active is None:
return
try:
argv = json.loads(CONFIG.read_text(encoding="utf-8"))["notify"]
NOTIFIED.mkdir(parents=True, exist_ok=True)
os.close(os.open(NOTIFIED / str(active["since"]), os.O_CREAT | os.O_EXCL | os.O_WRONLY))
subprocess.run(
[*argv, f"Agents radio MAYDAY by {active['by']}: {active['text']}"],
capture_output=True,
timeout=NOTIFY_SECONDS,
check=False,
)
except (OSError, ValueError, KeyError, TypeError, subprocess.TimeoutExpired):
return
def listening(sid: str) -> bool:
try:
pid = int((LISTEN / f"{sid}.pid").read_text(encoding="utf-8"))
except (OSError, ValueError):
return False
if pid <= 0:
return False
try:
os.kill(pid, 0)
except PermissionError:
return True
except OSError:
return False
return True
def unreached(cls: str, to: list[str], sender: str) -> list[str]:
live = council().live_sessions()
if cls == "mayday":
targets = [(name or sid, [sid]) for sid, name in live.items() if sid != sender]
else:
targets = [(name, [sid for sid, label in live.items() if name in (label, sid)]) for name in to]
return [name for name, sids in targets if not any(map(listening, sids))]
def cmd_send(args: argparse.Namespace) -> int:
to = args.to or []
if args.cls == "direct" and not to:
fail("direct needs --to")
if args.cls != "direct" and to:
fail(f"{args.cls} goes to all; drop --to")
if args.wake and args.cls != "direct":
fail("--wake only with direct")
text = " ".join(args.text.split())
if not text:
fail(f"{args.cls} needs a text")
sid, _ = council().caller(args.session)
sender = name_of(sid)
ROOT.mkdir(parents=True, exist_ok=True)
with LOCK.open("w") as lock:
fcntl.flock(lock, fcntl.LOCK_EX)
rows = items()
ts = time.time()
if rows:
ts = max(ts, math.nextafter(max(row["ts"] for row in rows), math.inf))
entry = {"ts": ts, "from": sender, "to": to, "text": text, "class": args.cls}
if args.wake:
entry["wake"] = True
write_atomic(NEWS, "".join(json.dumps(row) + "\n" for row in [*rows[-(KEPT - 1):], entry]))
if args.cls == "mayday":
write_atomic(
SILENCE,
json.dumps({"since": ts, "until": ts + SILENCE_SECONDS, "by": sender, "text": text}),
)
print(f"sent {args.cls} to {', '.join(to) or 'all'}")
if args.wake or args.cls == "mayday":
for name in unreached(args.cls, to, sid):
print(f"wake: SendMessage to {name}")
return 0
def cmd_post(args: argparse.Namespace) -> int:
args.cls = "direct" if args.to else "broadcast"
args.wake = False
return cmd_send(args)
def cmd_clear(args: argparse.Namespace) -> int:
sid, _ = council().caller(args.session)
me = name_of(sid)
ROOT.mkdir(parents=True, exist_ok=True)
with LOCK.open("w") as lock:
fcntl.flock(lock, fcntl.LOCK_EX)
active = silence()
if active is None:
print("no radio silence active")
return 0
if active["by"] != me and not args.force:
print(f"radio silence held by {active['by']}; only they clear it, anyone else with --force")
return 3
SILENCE.unlink(missing_ok=True)
forced = f", forced by {me}" if active["by"] != me else ""
print(f"cleared radio silence by {active['by']}{forced}")
return 0
def unseen(sid: str, name: str) -> list[dict]:
floor = max(prompt_mark(sid), time.time() - WINDOW)
return [row for row in items() if row["ts"] > floor and for_me(row, name)]
def delivered(row: dict, state: dict) -> bool:
if urgent(row):
return row["ts"] <= mark(state, "urgent")
return bool(row.get("wake")) and row["ts"] <= mark(state, "listen")
def cmd_hook(_: argparse.Namespace) -> int:
try:
sid = hook_session()
if not sid:
return 0
active = silence()
out = [silence_line(active)] if active else []
name = name_of(sid)
with session(sid) as state:
rows = unseen(sid, name)
fresh = [row for row in rows if not delivered(row, state)]
picked = sorted(fresh, key=lambda row: (rank(row), -row["ts"]))[:SHOWN]
out += [line(row) for row in ordered(picked)]
if len(fresh) > SHOWN:
out.append(f"RADIO {len(fresh) - SHOWN} more items: radio.py show --all")
if rows:
SEEN.mkdir(parents=True, exist_ok=True)
(SEEN / sid).write_text(str(max(row["ts"] for row in rows)), encoding="utf-8")
if out:
print("\n".join(out))
notify_once(active)
except Exception: # noqa: BLE001
return 0
return 0
def cmd_tool(_: argparse.Namespace) -> int:
try:
sid = hook_session()
if not sid:
return 0
with session(sid) as state:
mtime = news_mtime()
if mtime is None or mtime <= mark(state, "mtime"):
return 0
name = name_of(sid)
floor = max(mark(state, "urgent"), prompt_mark(sid), time.time() - WINDOW)
rows = ordered([row for row in items() if urgent(row) and row["ts"] > floor and for_me(row, name)])
state["mtime"] = mtime
if rows:
state["urgent"] = max(row["ts"] for row in rows)
save(sid, state)
if rows:
context = "\n".join(line(row) for row in rows)
print(json.dumps({"hookSpecificOutput": {"hookEventName": "PostToolBatch", "additionalContext": context}}))
if any(kind(row) == "mayday" for row in rows):
notify_once(silence())
except Exception: # noqa: BLE001
return 0
return 0
def calls(sid: str) -> list[str]:
name = name_of(sid)
with session(sid) as state:
floor = max(prompt_mark(sid), time.time() - WINDOW)
heard = max(mark(state, "listen"), floor)
loud = max(mark(state, "urgent"), floor)
rows = [row for row in items() if row["ts"] > floor and for_me(row, name)]
wakes = [
row
for row in rows
if row["ts"] > heard
and ((kind(row) == "direct" and row.get("wake")) or (kind(row) == "mayday" and row["ts"] > loud))
]
if not wakes:
return []
alarms = [row for row in rows if urgent(row) and row["ts"] > loud]
state["listen"] = max(row["ts"] for row in wakes)
if alarms:
state["urgent"] = max(row["ts"] for row in alarms)
save(sid, state)
return [line(row) for row in ordered(alarms + [row for row in wakes if not urgent(row)])]
def cmd_listen(_: argparse.Namespace) -> int:
try:
sid = hook_session()
if not sid:
return 0
LISTEN.mkdir(parents=True, exist_ok=True)
pidfile = LISTEN / f"{sid}.pid"
mine = str(os.getpid())
write_atomic(pidfile, mine)
parent = os.getppid()
known: float | None = None
# A listener outliving its session would poll forever; the session exiting changes its parent.
while pidfile.read_text(encoding="utf-8") == mine and os.getppid() == parent:
mtime = news_mtime()
if mtime != known:
known = mtime
lines = calls(sid)
if lines:
print("\n".join(lines), file=sys.stderr)
notify_once(silence())
return 2
time.sleep(POLL_SECONDS)
except Exception: # noqa: BLE001
return 0
return 0
def cmd_show(args: argparse.Namespace) -> int:
active = silence()
if active:
print(silence_line(active))
if args.all:
rows = items()
else:
sid, _ = council().caller(args.session)
rows = ordered(unseen(sid, name_of(sid)))
for row in rows:
print(line(row))
return 0
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
sub = parser.add_subparsers(dest="command", required=True)
p = sub.add_parser("send")
p.add_argument("cls", metavar="class", choices=CLASSES)
p.add_argument("text")
p.add_argument("--to", action="append")
p.add_argument("--wake", action="store_true")
p.add_argument("--session")
p.set_defaults(func=cmd_send)
p = sub.add_parser("post")
p.add_argument("text")
p.add_argument("--to", action="append")
p.add_argument("--session")
p.set_defaults(func=cmd_post)
p = sub.add_parser("clear")
p.add_argument("--force", action="store_true")
p.add_argument("--session")
p.set_defaults(func=cmd_clear)
sub.add_parser("hook").set_defaults(func=cmd_hook)
sub.add_parser("tool").set_defaults(func=cmd_tool)
sub.add_parser("listen").set_defaults(func=cmd_listen)
p = sub.add_parser("show")
p.add_argument("--all", action="store_true")
p.add_argument("--session")
p.set_defaults(func=cmd_show)
args = parser.parse_args()
return args.func(args)
if __name__ == "__main__":
sys.exit(main())
+54 -19
View File
@@ -145,22 +145,59 @@ class TestSettingsRegistration(unittest.TestCase):
)
return json.loads(self.settings.read_text(encoding="utf-8"))
@staticmethod
def _hooks(data: dict, event: str, script: str) -> list[dict]:
return [
hook
for entry in data["hooks"].get(event, [])
for hook in entry.get("hooks", [])
if script in hook.get("command", "")
]
def test_hook_registered_once(self):
self._patch()
data = self._patch()
entries = json.dumps(data["hooks"]["UserPromptSubmit"])
self.assertEqual(entries.count("kvb-dream/hooks/reminder.sh"), 1)
self.assertEqual(entries.count("kvb-coach/hooks/reminder.sh"), 1)
self.assertNotIn("tune/hooks/reminder.sh", entries)
gate = json.dumps(data["hooks"]["PreToolUse"])
self.assertEqual(gate.count("kvb-org-organize/scripts/council.py"), 1)
self.assertEqual(gate.count("kvb-org-organize/scripts/watch.py"), 1)
self.assertEqual(entries.count("kvb-org-organize/scripts/watch.py"), 1)
self.assertEqual(entries.count("kvb-org-organize/scripts/news.py"), 1)
post = json.dumps(data["hooks"]["PostToolUse"])
self.assertEqual(post.count("kvb-org-organize/scripts/watch.py"), 1)
self.assertEqual(gate.count("kvb-org-perform/scripts/log.py"), 1)
self.assertEqual(post.count("kvb-org-perform/scripts/log.py"), 1)
expected = {
("UserPromptSubmit", "kvb-dream/hooks/reminder.sh"): 1,
("UserPromptSubmit", "kvb-coach/hooks/reminder.sh"): 1,
("UserPromptSubmit", "tune/hooks/reminder.sh"): 0,
("UserPromptSubmit", "kvb-org-organize/scripts/watch.py"): 1,
("UserPromptSubmit", "kvb-org-radio/scripts/radio.py"): 1,
("UserPromptSubmit", "kvb-org-organize/scripts/news.py"): 0,
("PostToolBatch", "kvb-org-radio/scripts/radio.py"): 1,
("Stop", "kvb-org-radio/scripts/radio.py"): 1,
("PreToolUse", "kvb-org-organize/scripts/council.py"): 1,
("PreToolUse", "kvb-org-organize/scripts/watch.py"): 1,
("PreToolUse", "kvb-org-queue/scripts/queue.py"): 1,
("PreToolUse", "kvb-org-perform/scripts/log.py"): 1,
("PostToolUse", "kvb-org-organize/scripts/watch.py"): 1,
("PostToolUse", "kvb-org-perform/scripts/log.py"): 1,
}
for (event, script), count in expected.items():
with self.subTest(event=event, script=script):
self.assertEqual(len(self._hooks(data, event, script)), count)
def test_radio_listener_wakes_asynchronously_and_hooks_survive_a_missing_script(self):
data = self._patch()
(listen,) = self._hooks(data, "Stop", "kvb-org-radio/scripts/radio.py")
self.assertIs(listen["asyncRewake"], True)
self.assertGreater(listen["timeout"], 3600)
self.assertTrue(listen["command"].endswith(" listen"))
(gate,) = self._hooks(data, "PreToolUse", "kvb-org-queue/scripts/queue.py")
result = subprocess.run(["sh", "-c", gate["command"]], capture_output=True, check=False)
self.assertEqual(result.returncode, 1)
def test_news_hook_moves_to_radio(self):
base = Path(self.tmp.name) / "skills"
self.settings.write_text(
json.dumps({"hooks": {"UserPromptSubmit": [
{"hooks": [{"type": "command", "command": f'python3 "{base}/kvb-org-organize/scripts/news.py" hook'}]}
]}}),
encoding="utf-8",
)
data = self._patch()
self.assertEqual(self._hooks(data, "UserPromptSubmit", "kvb-org-organize/scripts/news.py"), [])
self.assertEqual(len(self._hooks(data, "UserPromptSubmit", "kvb-org-radio/scripts/radio.py")), 1)
def test_code_graph_plugin_is_retired_for_the_agent_agnostic_installer(self):
self.settings.write_text(
@@ -231,12 +268,10 @@ class TestSettingsRegistration(unittest.TestCase):
for old in ("/autotune/", "/kvb-skill-tune/", "/kvb-skill-dream/", "/democracy/", "/subagent-model/"):
self.assertNotIn(f"skills{old}", text)
self.assertIn("echo keep", text)
entries = json.dumps(data["hooks"]["UserPromptSubmit"])
self.assertEqual(entries.count("kvb-dream/hooks/reminder.sh"), 1)
self.assertEqual(entries.count("kvb-org-organize/scripts/watch.py"), 1)
gate = json.dumps(data["hooks"]["PreToolUse"])
self.assertEqual(gate.count("kvb-org-organize/scripts/council.py"), 1)
self.assertEqual(gate.count("kvb-org-perform/scripts/log.py"), 1)
self.assertEqual(len(self._hooks(data, "UserPromptSubmit", "kvb-dream/hooks/reminder.sh")), 1)
self.assertEqual(len(self._hooks(data, "UserPromptSubmit", "kvb-org-organize/scripts/watch.py")), 1)
self.assertEqual(len(self._hooks(data, "PreToolUse", "kvb-org-organize/scripts/council.py")), 1)
self.assertEqual(len(self._hooks(data, "PreToolUse", "kvb-org-perform/scripts/log.py")), 1)
if __name__ == "__main__":
+38 -15
View File
@@ -1,8 +1,9 @@
"""Exercise news.py: post, the hook's once-only delivery, addressing, own posts, and a broken hook input."""
"""Exercise the news.py shim: it forwards post, hook and show to radio.py, and a missing radio.py never blocks a prompt."""
from __future__ import annotations
import json
import shutil
import subprocess
import sys
import tempfile
@@ -17,44 +18,66 @@ class TestNews(unittest.TestCase):
def setUp(self):
tmp = tempfile.TemporaryDirectory()
self.addCleanup(tmp.cleanup)
self.home = Path(tmp.name) / "home"
self.root = Path(tmp.name) / "council"
self.tmp = Path(tmp.name)
self.home = self.tmp / "home"
self.root = self.tmp / "council"
sessions = self.home / ".claude" / "sessions"
sessions.mkdir(parents=True)
for sid, name in (("s1", "One"), ("s2", "Two"), ("s3", "Three")):
(sessions / f"{sid}.json").write_text(json.dumps({"sessionId": sid, "name": name}), encoding="utf-8")
def _run(self, *args: str, sid: str | None = "s1", stdin: str = "") -> subprocess.CompletedProcess:
env = {"PATH": "/usr/bin:/bin", "HOME": str(self.home), "AGENTS_COUNCIL_ROOT": str(self.root)}
def _exec(self, news: Path, sid: str | None, stdin: str, args: tuple[str, ...]) -> subprocess.CompletedProcess:
env = {
"PATH": "/usr/bin:/bin",
"HOME": str(self.home),
"AGENTS_COUNCIL_ROOT": str(self.root),
"AGENTS_COUNCIL_CONFIG": str(self.tmp / "config.json"),
}
if sid:
env["CLAUDE_CODE_SESSION_ID"] = sid
return subprocess.run(
[sys.executable, str(NEWS), *args], input=stdin, capture_output=True, text=True, env=env, check=False
[sys.executable, str(news), *args], input=stdin, capture_output=True, text=True, env=env, check=False
)
def _cli(self, sid: str | None, *args: str) -> subprocess.CompletedProcess:
return self._exec(NEWS, sid, "", args)
def _hook(self, sid: str) -> list[str]:
result = self._run("hook", sid=None, stdin=json.dumps({"session_id": sid, "prompt": "x"}))
result = self._exec(NEWS, None, json.dumps({"session_id": sid, "prompt": "x"}), ("hook",))
self.assertEqual(result.returncode, 0, result.stderr)
return [line.split(" ", 2)[2] for line in result.stdout.splitlines()]
return [line.split(" ", 3)[3] for line in result.stdout.splitlines()]
def test_every_session_sees_a_post_once_except_its_author(self):
self.assertEqual(self._run("post", "core #1358 merged").stdout, "posted to all\n")
self.assertEqual(self._cli("s1", "post", "core #1358 merged").stdout, "sent broadcast to all\n")
self.assertEqual(self._hook("s2"), ["One: core #1358 merged"])
self.assertEqual(self._hook("s2"), [])
self.assertEqual(self._hook("s3"), ["One: core #1358 merged"])
self.assertEqual(self._hook("s1"), [])
def test_addressed_post_reaches_only_its_target(self):
self._run("post", "your gate is granted", "--to", "Two")
self._cli("s1", "post", "your gate is granted", "--to", "Two")
self.assertEqual(self._hook("s3"), [])
self.assertEqual(self._hook("s2"), ["One to Two: your gate is granted"])
self.assertEqual(len(self._run("show", "--all").stdout.splitlines()), 1)
self.assertEqual(len(self._cli("s1", "show", "--all").stdout.splitlines()), 1)
def test_hook_never_fails_and_post_needs_a_session(self):
self.assertEqual((self._run("hook", sid=None, stdin="not json").returncode), 0)
self.assertEqual(self._run("hook", sid=None, stdin="{}").stdout, "")
self.assertEqual(self._run("post", "x", sid=None).returncode, 2)
self.assertEqual(self._run("post", " ").returncode, 2)
self.assertEqual(self._exec(NEWS, None, "not json", ("hook",)).returncode, 0)
self.assertEqual(self._exec(NEWS, None, "{}", ("hook",)).stdout, "")
self.assertEqual(self._cli(None, "post", "x").returncode, 2)
self.assertEqual(self._cli("s1", "post", " ").returncode, 2)
def test_without_radio_the_hook_passes_and_the_rest_fail(self):
lone = self.tmp / "skills" / "kvb-org-organize" / "scripts" / "news.py"
lone.parent.mkdir(parents=True)
shutil.copy(NEWS, lone)
hook = self._exec(lone, None, json.dumps({"session_id": "s2"}), ("hook",))
self.assertEqual((hook.returncode, hook.stdout), (0, ""))
for args in (("post", "x"), ("show",)):
with self.subTest(args=args):
result = self._exec(lone, "s1", "", args)
self.assertEqual(result.returncode, 2)
self.assertIn("kvb-org-radio", result.stderr)
self.assertFalse(self.root.exists())
if __name__ == "__main__":
+503
View File
@@ -0,0 +1,503 @@
"""Exercise the kvb-org-queue grant order, its history learning and its hold, list, history, cancel and block commands."""
from __future__ import annotations
import importlib.util
import json
import os
import signal
import subprocess
import sys
import tempfile
import time
import unittest
from collections.abc import Callable
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parents[1]
QUEUE = REPO_ROOT / "skills" / "kvb-org-queue" / "scripts" / "queue.py"
HISTORY = REPO_ROOT / "skills" / "kvb-org-queue" / "scripts" / "history.py"
NOW = 1_800_000_000.0
HOUR = 3600
BASE = {
"free_gb": 100.0,
"floor": 10.0,
"stage": "ok",
"silenced": False,
"slot_limits": {"test": 5, "deploy": 1},
"ranks": {"work/a": 0, "work/b": 1, "work/c": 2},
"now": NOW,
}
CHILD = (
"import json, os, sys, time\n"
"deadline = time.monotonic() + 20\n"
"while True:\n"
" with open(sys.argv[1], encoding='utf-8') as handle:\n"
" entries = json.load(handle)['entries']\n"
" if (entries and entries[0]['child'] == os.getpid()) or time.monotonic() > deadline:\n"
" break\n"
" time.sleep(0.05)\n"
"with open(sys.argv[2], 'w', encoding='utf-8') as out:\n"
" json.dump(entries, out)\n"
"sys.exit(7)\n"
)
def _load(path: Path):
spec = importlib.util.spec_from_file_location(f"{path.stem}_under_test", path)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
def history_row(seq: int, **fields: object) -> dict:
return {
"id": f"r{seq}", "kind": "test", "signature": "make test", "workdir": "/w", "start": NOW - 600, "end": NOW,
"minutes": 10.0, "exit": 0, "free_before_gb": 50.0, "min_free_gb": 49.0, "disk_gb": 1.0, "mem_before": 20.0,
"mem_max": 30.0, "cpu_before": 10.0, "cpu_max": 40.0, "concurrent": 0, **fields,
}
def job(seq: int, **fields: object) -> dict:
return {
"id": f"j{seq}", "seq": seq, "kind": "test", "task": "work/a", "need_gb": 1, "hotfix": None,
"workdir": f"/w{seq}", "status": "waiting", "enqueued": NOW, **fields,
}
def stored(seq: int, status: str, pid: int, pid_start: str | None, child: int | None,
child_start: str | None) -> dict:
return {
"id": f"test-{seq}", "seq": seq, "kind": "test", "task": "work/a", "need_gb": 0, "hotfix": None,
"workdir": f"/w{seq}", "session": "s1", "name": "s1", "pid": pid, "pid_start": pid_start,
"child": child, "child_start": child_start, "status": status, "enqueued": time.time(),
"granted": time.time() if status == "active" else None, "cmd": "true",
}
class TestEvaluate(unittest.TestCase):
@classmethod
def setUpClass(cls):
cls.queue = _load(QUEUE)
def decide(self, entries: list[dict], **overrides: object) -> dict:
return self.queue.evaluate(entries, **{**BASE, **overrides})
def test_hotfix_goes_first_then_rank_then_arrival_and_unknown_tasks_last(self):
entries = [
job(1, task="work/c"), job(2, task="work/gone"), job(3), job(4), job(5, task="work/c", hotfix="fix-1"),
]
self.assertEqual(list(self.decide(entries)), ["j5", "j3", "j4", "j1", "j2"])
def test_each_whole_hour_waited_moves_one_rank_up(self):
fresh = job(1)
aged = job(2, task="work/c", enqueued=NOW - 3 * HOUR)
almost = job(3, task="work/b", enqueued=NOW - HOUR + 1)
self.assertEqual(list(self.decide([fresh, aged, almost])), ["j2", "j1", "j3"])
tied = job(4, task="work/c", enqueued=NOW - 2 * HOUR)
self.assertEqual(list(self.decide([tied, fresh])), ["j1", "j4"])
def test_the_slot_limit_counts_active_jobs_per_kind(self):
verdicts = self.decide([job(1, kind="deploy", status="active"), job(2, kind="deploy"), job(3)])
self.assertEqual(verdicts, {"j2": "all 1 deploy slots taken", "j3": None})
def test_a_higher_waiter_is_granted_virtually_before_a_lower_one(self):
verdicts = self.decide([job(1, task="work/b", kind="deploy"), job(2, kind="deploy")])
self.assertEqual(verdicts, {"j2": None, "j1": "all 1 deploy slots taken"})
def test_disk_reservation_keeps_smaller_jobs_from_starving_a_big_one(self):
big, small = job(1, need_gb=50), job(2, task="work/b", need_gb=5)
verdicts = self.decide([big, small], free_gb=40.0)
self.assertEqual(verdicts["j1"], "50 GB would cut into the 10 GB floor; reserved")
self.assertEqual(verdicts["j2"], "5 GB would cut into the 10 GB floor; reserved")
self.assertIsNone(self.decide([small], free_gb=40.0)["j2"])
running = job(3, need_gb=25, status="active")
self.assertIsNone(self.decide([running, job(4, need_gb=5)], free_gb=40.0)["j4"])
self.assertIsNotNone(self.decide([running, job(4, need_gb=6)], free_gb=40.0)["j4"])
def test_warn_admits_only_jobs_without_disk_and_stop_admits_none(self):
entries = [job(1, need_gb=0), job(2, task="work/b")]
verdicts = self.decide(entries, stage="warn")
self.assertEqual(verdicts, {"j1": None, "j2": "disk stage warn admits only jobs that need no disk"})
for stage in ("stop", "critical"):
with self.subTest(stage=stage):
self.assertEqual(set(self.decide(entries, stage=stage).values()), {f"disk stage {stage}"})
def test_radio_silence_blocks_every_job(self):
verdicts = self.decide([job(1, need_gb=0), job(2)], silenced=True)
self.assertEqual(set(verdicts.values()), {"radio silence"})
def test_a_busy_workdir_blocks_a_second_job_there(self):
verdicts = self.decide([
job(1, workdir="/w", status="active"), job(2, workdir="/w"), job(3, workdir="/x"),
job(4, task="work/b", workdir="/x"),
])
self.assertEqual(verdicts, {"j2": "/w runs another job", "j3": None, "j4": "/x runs another job"})
class TestHistory(unittest.TestCase):
@classmethod
def setUpClass(cls):
cls.history = _load(HISTORY)
def learn(self, rows: list[dict]) -> tuple[dict, dict]:
return self.history.learn(rows, NOW, {"load_limits": {"mem_percent": 80, "cpu_percent": 90}})
def test_signature_names_a_command_apart_from_paths_options_and_variables(self):
cases = {
("make", "-C", "/repo", "-j4", "VAR=1", "compose-deploy", "apps=x"): "make compose-deploy",
("/usr/bin/make", "-j", "8", "-f", "Other.mk", "test"): "make test",
("make", "-k"): "make",
("python3", "/s/kvb-check-gate/scripts/gate.py", "run", "--repo", ".", "--target", "test"): "gate.py run",
("/usr/bin/python3.14", "-u", "x.py", "--flag", "sub"): "x.py sub",
("python3", "-m", "unittest", "tests.test_queue"): "unittest tests.test_queue",
("python3", "-c", "print(1)"): "python3 -c",
("bash", "scripts/install.sh"): "install.sh",
("docker", "compose", "up", "-d"): "docker compose",
("/bin/true",): "true",
}
for argv, expected in cases.items():
with self.subTest(argv=argv):
self.assertEqual(self.history.signature(list(argv)), expected)
def test_learn_waits_for_min_runs_inside_the_30_day_window(self):
four = [history_row(n) for n in range(4)]
signatures, kinds = self.learn(four)
self.assertEqual((signatures["make test"]["runs"], signatures["make test"]["need_gb"]), (4, None))
self.assertEqual((kinds["test"]["runs"], kinds["test"]["limit"]), (4, None))
stale = history_row(4, end=NOW - self.history.WINDOW_SECONDS - 1)
self.assertIsNone(self.learn([*four, stale])[0]["make test"]["need_gb"])
signatures, kinds = self.learn([*four, history_row(4, end=NOW - self.history.WINDOW_SECONDS)])
self.assertEqual((signatures["make test"]["need_gb"], signatures["make test"]["minutes"]), (2, 10))
self.assertEqual(kinds["test"]["limit"], 2)
def test_learn_needs_the_p90_disk_plus_one_gb_and_takes_the_median_minutes(self):
runs = zip((4, 1, 10, 3, 2), (5, 9, 7, 1, 30))
stats = self.learn([history_row(n, disk_gb=gb, minutes=m) for n, (gb, m) in enumerate(runs)])[0]["make test"]
self.assertAlmostEqual(stats["disk_gb"], 7.6)
self.assertEqual((stats["need_gb"], stats["minutes"]), (9, 7))
def test_learn_fits_the_slot_limit_between_the_median_load_and_the_load_limits(self):
loads = ((10, 5), (20, 5), (20, 10), (30, 15), (40, 15))
rows = [history_row(n, mem_before=before, mem_max=before + share) for n, (before, share) in enumerate(loads)]
kind = self.learn(rows)[1]["test"]
self.assertEqual((kind["ram_share"], kind["cpu_share"], kind["limit"]), (15, 30, 2))
idle_cpu = [{**row, "cpu_max": row["cpu_before"]} for row in rows]
self.assertEqual(self.learn(idle_cpu)[1]["test"]["limit"], 4)
idle = [{**row, "mem_max": row["mem_before"]} for row in idle_cpu]
self.assertEqual(self.learn(idle)[1]["test"]["limit"], 60)
crowded = [history_row(n, mem_before=90, mem_max=95) for n in range(5)]
self.assertEqual(self.learn(crowded)[1]["test"]["limit"], 1)
def test_slot_limits_take_the_learned_limit_else_the_seed_and_a_smaller_cap(self):
kinds = {"deploy": {"limit": 3}, "build": {"limit": None}}
self.assertEqual(self.history.slot_limits(kinds, {"deploy": 2, "swarm": 0, "test": 9}, ["test", "gate"]), {
"build": (1, "seed"), "deploy": (2, "capped by config"), "gate": (1, "seed"),
"swarm": (0, "capped by config"), "test": (1, "seed"),
})
self.assertEqual(self.history.slot_limits(kinds, {"deploy": 5}, [])["deploy"], (3, "learned"))
def test_row_keeps_the_extremes_and_floors_the_disk_taken_at_zero(self):
samples = [
{"free_gb": 50, "mem": 20, "cpu": 5}, {"free_gb": 47.5, "mem": 60, "cpu": 90},
{"free_gb": 49, "mem": 30, "cpu": 1},
]
row = self.history.row({"id": "j"}, NOW, NOW + 90, 3, samples, 2)
self.assertEqual(row, {
"id": "j", "start": NOW, "end": NOW + 90, "minutes": 1.5, "exit": 3, "free_before_gb": 50,
"min_free_gb": 47.5, "disk_gb": 2.5, "mem_before": 20, "mem_max": 60, "cpu_before": 5, "cpu_max": 90,
"concurrent": 2,
})
grown = self.history.row({"id": "j"}, NOW, NOW, 0, [samples[0], {**samples[0], "free_gb": 55}], 0)
self.assertEqual(grown["disk_gb"], 0)
def test_read_merges_the_files_by_id_and_skips_broken_lines(self):
with tempfile.TemporaryDirectory() as tmp:
first, second = Path(tmp) / "a.jsonl", Path(tmp) / "b.jsonl"
first.write_text('{"id": "x", "v": 1}\n{broken\n[1]\n', encoding="utf-8")
second.write_text('{"id": "x", "v": 2}\n{"id": "y"}\n', encoding="utf-8")
rows = self.history.read([first, second, Path(tmp) / "missing.jsonl"])
self.assertEqual(rows, [{"id": "x", "v": 1}, {"id": "y"}])
def test_eta_adds_the_remaining_and_the_waiting_median_minutes(self):
signatures = self.learn([history_row(n) for n in range(5)])[0]
eta = self.history.eta
self.assertEqual(eta(signatures, [("make test", NOW - 240)], ["make test"], NOW), 16)
self.assertEqual(eta(signatures, [("make test", NOW - 900)], [], NOW), 0)
self.assertEqual(eta(signatures, [], [], NOW), 0)
self.assertIsNone(eta(signatures, [], ["make test", "true"], NOW))
class TestQueueCommands(unittest.TestCase):
def setUp(self):
tmp = tempfile.TemporaryDirectory()
self.addCleanup(tmp.cleanup)
self.base = Path(tmp.name).resolve()
self.root = self.base / "council"
self.root.mkdir()
home = self.base / "home"
home.mkdir()
self.history_file = self.root / "queue-history.jsonl"
self.kept = home / ".local" / "state" / "agents-council" / "queue-history.jsonl"
self.config = self.base / "config.json"
self.cfg = {
"disk": str(self.base),
"thresholds_gb": {"warn": 0, "stop": 0, "critical": 0},
"projection_minutes": 30,
"slot_need_gb": {"test": 0.001, "deploy": 0},
"slot_limits": {"test": 1, "deploy": 1},
"load_limits": {"cpu_percent": 101, "mem_percent": 101, "sample_seconds": 0.1, "kinds": ["test"]},
"heavy_commands": ["^make compose-deploy"],
"never_block": ["^make compose-down"],
}
self.write_config()
(self.root / "rasci.json").write_text(json.dumps({
"work/a": {"roles": {}, "status": "running"},
"work/done": {"roles": {}, "status": "done"},
"hotfix/fix-1": {"roles": {}, "status": "running", "stage": "pr"},
}), encoding="utf-8")
self.env = {
"PATH": os.environ["PATH"],
"HOME": str(home),
"GIT_CEILING_DIRECTORIES": str(self.base.parent),
"AGENTS_COUNCIL_ROOT": str(self.root),
"AGENTS_COUNCIL_CONFIG": str(self.config),
}
def write_config(self) -> None:
self.config.write_text(json.dumps(self.cfg), encoding="utf-8")
def write_store(self, entries: list[dict]) -> None:
(self.root / "queue.json").write_text(json.dumps({"seq": len(entries), "entries": entries}), encoding="utf-8")
def entries(self) -> list[dict]:
return json.loads((self.root / "queue.json").read_text(encoding="utf-8"))["entries"]
def write_history(self, path: Path, rows: list[dict]) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text("".join(json.dumps(row) + "\n" for row in rows), encoding="utf-8")
def blocked_hold(self, *command: str) -> tuple[subprocess.Popen, Path]:
err = self.base / "hold.err"
with open(err, "w", encoding="utf-8") as handle:
hold = subprocess.Popen(
[sys.executable, str(QUEUE), "hold", "--kind", "deploy", "--task", "work/a", "--session", "s1",
"--", *command],
cwd=self.base, env=self.env, stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL, stderr=handle,
)
self.addCleanup(hold.wait)
self.addCleanup(hold.kill)
self.wait_for(lambda: "waits at position 1" in err.read_text(encoding="utf-8"))
return hold, err
def run_queue(self, *args: str) -> subprocess.CompletedProcess:
return subprocess.run(
[sys.executable, str(QUEUE), *args], cwd=self.base, env=self.env, stdin=subprocess.DEVNULL,
capture_output=True, text=True, timeout=60, check=False,
)
def hook(self, payload: str) -> subprocess.CompletedProcess:
return subprocess.run(
[sys.executable, str(QUEUE), "block"], cwd=self.base, env=self.env, input=payload,
capture_output=True, text=True, timeout=60, check=False,
)
def bash(self, command: str) -> int:
return self.hook(json.dumps({"tool_input": {"command": command}})).returncode
def wait_for(self, condition: Callable[[], bool]) -> None:
deadline = time.monotonic() + 30
while not condition():
self.assertLess(time.monotonic(), deadline)
time.sleep(0.05)
def test_hold_runs_the_command_on_its_grant_passes_its_exit_code_and_leaves(self):
seen = self.base / "seen.json"
result = self.run_queue(
"hold", "--kind", "test", "--task", "work/a", "--hotfix", "fix-1", "--session", "s1", "--",
sys.executable, "-c", CHILD, str(self.root / "queue.json"), str(seen),
)
self.assertEqual((result.returncode, result.stderr), (7, ""))
[entry] = json.loads(seen.read_text(encoding="utf-8"))
self.assertEqual(
[entry[k] for k in ("id", "status", "kind", "task", "need_gb", "hotfix", "workdir", "session")],
["test-1", "active", "test", "work/a", 0.001, "fix-1", str(self.base), "s1"],
)
self.assertEqual(self.entries(), [])
[row] = [json.loads(line) for line in self.history_file.read_text(encoding="utf-8").splitlines()]
self.assertEqual(
[row[k] for k in ("kind", "signature", "workdir", "exit", "concurrent")],
["test", f"{Path(sys.executable).name} -c", str(self.base), 7, 0],
)
self.assertTrue(row["id"].startswith("test-1-"))
self.assertLessEqual(row["start"], row["end"])
self.assertEqual(row["disk_gb"], round(max(0.0, row["free_before_gb"] - row["min_free_gb"]), 2))
self.assertLessEqual(row["mem_before"], row["mem_max"])
def test_hold_runs_without_slot_limits_on_the_seed_limit(self):
del self.cfg["slot_limits"]
self.write_config()
result = self.run_queue("hold", "--kind", "test", "--task", "work/a", "--session", "s1", "--", "true")
self.assertEqual((result.returncode, result.stderr), (0, ""))
def test_a_blocked_hold_reports_once_shows_in_list_and_leaves_on_sigterm(self):
self.cfg["slot_limits"]["deploy"] = 0
self.write_config()
hold, err = self.blocked_hold("true")
listing = self.run_queue("list")
self.assertEqual(listing.returncode, 0, listing.stderr)
row = ["1", "deploy-1", "waiting", "deploy", "work/a", "0", "0", "0", str(self.base), "all 0 deploy slots taken"]
self.assertEqual(listing.stdout.splitlines()[1], "\t".join(row))
hold.send_signal(signal.SIGTERM)
self.assertEqual(hold.wait(timeout=30), 128 + signal.SIGTERM)
self.assertEqual(
err.read_text(encoding="utf-8"), "queue: deploy-1 waits at position 1: all 0 deploy slots taken\n"
)
self.assertEqual(self.entries(), [])
def test_hold_refuses_bad_input_with_exit_2_and_enqueues_nothing(self):
def hold(*args: str) -> list[str]:
return ["hold", "--session", "s1", *args]
cases = (
("unknown kind", hold("--kind", "nope", "--task", "work/a", "--", "true"), "kind 'nope'"),
("unknown task", hold("--kind", "test", "--task", "work/zzz", "--", "true"), "no open task"),
("ended task", hold("--kind", "test", "--task", "work/done", "--", "true"), "no open task"),
("missing hotfix row", hold("--kind", "test", "--task", "work/a", "--hotfix", "nope", "--", "true"),
"hotfix/nope"),
("negative need", hold("--kind", "test", "--task", "work/a", "--need-gb", "-1", "--", "true"),
"--need-gb"),
("no command", hold("--kind", "test", "--task", "work/a"), "required"),
)
for name, args, message in cases:
with self.subTest(name):
result = self.run_queue(*args)
self.assertEqual(result.returncode, 2, result.stderr)
self.assertIn(message, result.stderr)
self.config.unlink()
missing_config = self.run_queue(*hold("--kind", "test", "--task", "work/a", "--", "true"))
self.assertEqual(missing_config.returncode, 2)
self.assertIn("no config", missing_config.stderr)
self.assertFalse((self.root / "queue.json").exists())
def test_list_drops_entries_of_dead_holders(self):
queue = _load(QUEUE)
gone = subprocess.Popen(["true"])
gone.wait()
me, mine = os.getpid(), queue.identity(os.getpid())
self.write_store([
stored(1, "waiting", gone.pid, "1", None, None),
stored(2, "waiting", me, "0", None, None),
stored(3, "active", gone.pid, "1", gone.pid, "1"),
stored(4, "active", gone.pid, "1", me, mine),
stored(5, "waiting", me, mine, None, None),
])
listing = self.run_queue("list")
self.assertEqual(listing.returncode, 0, listing.stderr)
self.assertEqual([e["id"] for e in self.entries()], ["test-4", "test-5"])
self.assertEqual(
[line.split("\t")[:3] for line in listing.stdout.splitlines()[1:]],
[["run", "test-4", "active"], ["1", "test-5", "waiting"]],
)
def test_hold_takes_the_need_learned_for_its_command_from_both_history_files(self):
self.cfg["slot_limits"]["deploy"] = 0
self.write_config()
now = time.time()
rows = [
history_row(n, kind="deploy", signature="true", end=now, disk_gb=gb) for n, gb in enumerate((4, 1, 10, 3, 2))
]
self.write_history(self.history_file, rows[:3])
self.write_history(self.kept, [{**rows[0], "disk_gb": 100}, *rows[3:]])
hold, _ = self.blocked_hold("true")
listing = self.run_queue("list")
self.assertEqual(listing.returncode, 0, listing.stderr)
self.assertEqual(listing.stdout.splitlines()[1].split("\t")[5], "9")
hold.send_signal(signal.SIGTERM)
self.assertEqual(hold.wait(timeout=30), 128 + signal.SIGTERM)
def test_list_estimates_from_the_learned_median_minutes(self):
queue = _load(QUEUE)
me, mine = os.getpid(), queue.identity(os.getpid())
now = time.time()
self.write_history(
self.history_file, [history_row(n, signature="make compose-deploy", end=now) for n in range(5)]
)
deploy = {"cmd": "make compose-deploy apps=x"}
self.write_store([
{**stored(1, "active", me, mine, me, mine), **deploy, "granted": now - 240},
{**stored(2, "waiting", me, mine, None, None), **deploy},
{**stored(3, "waiting", me, mine, None, None), **deploy},
stored(4, "waiting", me, mine, None, None),
stored(5, "waiting", me, mine, None, None),
])
listing = self.run_queue("list")
self.assertEqual(listing.returncode, 0, listing.stderr)
self.assertEqual([line.split("\t")[7] for line in listing.stdout.splitlines()], [
"eta_min", "6", "6", "16", "26", "-",
])
def test_history_prints_a_table_per_signature_and_per_kind(self):
self.cfg["slot_need_gb"]["build"] = 5
self.cfg["slot_limits"] = {"deploy": 9, "test": 0}
self.write_config()
now = time.time()
deploys = [
history_row(n, kind="deploy", signature="make compose-deploy", end=now, disk_gb=n + 1, minutes=10 + n,
cpu_before=1, cpu_max=21)
for n in range(5)
]
self.write_history(self.history_file, [*deploys, *(history_row(n, signature="true", end=now) for n in (5, 6))])
result = self.run_queue("history")
self.assertEqual(result.returncode, 0, result.stderr)
self.assertEqual(result.stdout, (
"| Signature | Kind | Runs | Median min | p90 disk GB | p90 RAM % | Need GB |\n"
"| --- | --- | --- | --- | --- | --- | --- |\n"
"| make compose-deploy | deploy | 5 | 12.0 | 4.6 | 10.0 | 6 |\n"
"| true | test | 2 | - | - | - | - |\n"
"\n"
"| Kind | Runs | p90 RAM share % | p90 CPU share % | Slot limit | Source |\n"
"| --- | --- | --- | --- | --- | --- |\n"
"| build | 0 | - | - | 1 | seed |\n"
"| deploy | 5 | 10.0 | 20.0 | 5 | learned |\n"
"| test | 2 | - | - | 0 | capped by config |\n"
))
def test_cancel_removes_only_the_callers_waiting_entry(self):
queue = _load(QUEUE)
me, mine = os.getpid(), queue.identity(os.getpid())
self.write_store([stored(1, "waiting", me, mine, None, None), stored(2, "active", me, mine, me, mine)])
self.assertEqual(self.run_queue("cancel", "test-1", "--session", "s2").returncode, 2)
self.assertEqual(self.run_queue("cancel", "test-2", "--session", "s1").returncode, 2)
result = self.run_queue("cancel", "test-1", "--session", "s1")
self.assertEqual((result.returncode, result.stdout), (0, "cancelled test-1\n"))
self.assertEqual([e["id"] for e in self.entries()], ["test-2"])
def test_block_allows_everything_without_queue_enforce(self):
self.assertEqual(self.bash("make compose-deploy"), 0)
self.cfg["queue_enforce"] = False
self.write_config()
self.assertEqual(self.bash("make compose-deploy"), 0)
def test_block_with_queue_enforce_refuses_heavy_commands_outside_the_queue(self):
self.cfg["queue_enforce"] = True
self.write_config()
refused = self.hook(json.dumps({"tool_input": {"command": "make compose-deploy apps=x"}}))
self.assertEqual(refused.returncode, 2)
self.assertIn("queue.py hold --kind <kind> --task <register task> -- <command>", refused.stderr)
self.assertEqual(self.bash("python3 skills/kvb-check-gate/scripts/gate.py run --repo . --target test"), 2)
for command in (
"python3 /x/queue.py hold --kind deploy --task work/a -- make compose-deploy",
"make compose-down",
"ls",
'git commit -m "make compose-deploy"',
):
with self.subTest(command=command):
self.assertEqual(self.bash(command), 0)
self.assertEqual(self.hook("not json").returncode, 0)
self.config.unlink()
self.assertEqual(self.bash("make compose-deploy"), 0)
if __name__ == "__main__":
unittest.main()
+319
View File
@@ -0,0 +1,319 @@
"""Exercise radio.py: class rules, the store shared with news.py, radio silence, the three hooks and the listener."""
from __future__ import annotations
import json
import os
import subprocess
import sys
import tempfile
import time
import unittest
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parents[1]
RADIO = REPO_ROOT / "skills" / "kvb-org-radio" / "scripts" / "radio.py"
CLASSES = ("BROADCAST", "DIRECT", "SECURITY", "PAN-PAN", "MAYDAY")
LISTENER = (
"import importlib.util, sys\n"
"spec = importlib.util.spec_from_file_location('radio', sys.argv[1])\n"
"radio = importlib.util.module_from_spec(spec)\n"
"spec.loader.exec_module(radio)\n"
"radio.POLL_SECONDS = 0.05\n"
"sys.argv = [sys.argv[1], 'listen']\n"
"sys.exit(radio.main())\n"
)
def strip(line: str) -> str:
parts = line.split(" ", 3)
if parts[0] != "RADIO" or parts[1] not in CLASSES:
return line
return f"{parts[1]} {parts[3]}"
class TestRadio(unittest.TestCase):
def setUp(self):
tmp = tempfile.TemporaryDirectory()
self.addCleanup(tmp.cleanup)
self.home = Path(tmp.name) / "home"
self.root = Path(tmp.name) / "council"
self.config = Path(tmp.name) / "config.json"
sessions = self.home / ".claude" / "sessions"
sessions.mkdir(parents=True)
for sid, name in (("s1", "One"), ("s2", "Two"), ("s3", "Three")):
(sessions / f"{sid}.json").write_text(json.dumps({"sessionId": sid, "name": name}), encoding="utf-8")
def _env(self, sid: str | None) -> dict[str, str]:
env = {
"PATH": "/usr/bin:/bin",
"HOME": str(self.home),
"AGENTS_COUNCIL_ROOT": str(self.root),
"AGENTS_COUNCIL_CONFIG": str(self.config),
}
if sid:
env["CLAUDE_CODE_SESSION_ID"] = sid
return env
def _exec(self, sid: str | None, stdin: str, args: tuple[str, ...]) -> subprocess.CompletedProcess:
return subprocess.run(
[sys.executable, str(RADIO), *args],
input=stdin, capture_output=True, text=True, env=self._env(sid), check=False, timeout=30,
)
def _cli(self, sid: str | None, *args: str) -> subprocess.CompletedProcess:
return self._exec(sid, "", args)
def _send(self, sid: str, *args: str) -> list[str]:
result = self._cli(sid, *args)
self.assertEqual(result.returncode, 0, result.stderr)
return result.stdout.splitlines()
def _hook(self, command: str, sid: str, **extra: str) -> str:
result = self._exec(None, json.dumps({"session_id": sid, **extra}), (command,))
self.assertEqual(result.returncode, 0, result.stderr)
return result.stdout
def _prompt(self, sid: str) -> list[str]:
return [strip(line) for line in self._hook("hook", sid).splitlines()]
def _tool(self, sid: str) -> list[str]:
out = self._hook("tool", sid)
if not out:
return []
output = json.loads(out)["hookSpecificOutput"]
self.assertEqual(output["hookEventName"], "PostToolBatch")
return [strip(line) for line in output["additionalContext"].splitlines()]
def _rows(self) -> list[dict]:
return [json.loads(line) for line in (self.root / "news.jsonl").read_text(encoding="utf-8").splitlines()]
def _pidfile(self, sid: str) -> Path:
return self.root / "radio-listen" / f"{sid}.pid"
def _listen(self, sid: str) -> subprocess.Popen:
proc = subprocess.Popen(
[sys.executable, "-c", LISTENER, str(RADIO)],
stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, env=self._env(None),
)
self.addCleanup(self._stop, proc)
proc.stdin.write(json.dumps({"session_id": sid}))
proc.stdin.close()
deadline = time.monotonic() + 10
while time.monotonic() < deadline and proc.poll() is None:
pidfile = self._pidfile(sid)
if pidfile.is_file() and pidfile.read_text(encoding="utf-8") == str(proc.pid):
return proc
time.sleep(0.02)
self.fail(f"listener for {sid} did not start")
def _stop(self, proc: subprocess.Popen) -> None:
if proc.poll() is None:
proc.kill()
proc.communicate()
def _finish(self, proc: subprocess.Popen) -> tuple[int, str]:
_, err = proc.communicate(timeout=10)
return proc.returncode, err
def test_classes_are_validated(self):
for args in (
("send", "broadcast", "x", "--to", "Two"),
("send", "direct", "x"),
("send", "mayday", "x", "--to", "Two"),
("send", "security", "x", "--wake"),
("send", "broadcast", "x", "--wake"),
("send", "bogus", "x"),
("send", "pan-pan", " "),
):
with self.subTest(args=args):
self.assertEqual(self._cli("s1", *args).returncode, 2)
self.assertEqual(self._cli(None, "send", "broadcast", "x").returncode, 2)
self.assertEqual(
self._send("s1", "send", "direct", "a b", "--to", "Two", "--to", "Three"), ["sent direct to Two, Three"]
)
self.assertEqual(self._send("s1", "post", "board"), ["sent broadcast to all"])
self.assertEqual(self._send("s1", "post", "one peer", "--to", "Two"), ["sent direct to Two"])
self.assertEqual(
[(r["class"], r["to"], r["text"], "wake" in r) for r in self._rows()],
[("direct", ["Two", "Three"], "a b", False), ("broadcast", [], "board", False), ("direct", ["Two"], "one peer", False)],
)
def test_old_rows_are_read_and_new_rows_keep_the_old_keys(self):
self.root.mkdir()
now = time.time()
old = [
{"ts": now - 10, "from": "Three", "to": [], "text": "old board"},
{"ts": now - 5, "from": "Three", "to": ["Two"], "text": "old direct"},
]
(self.root / "news.jsonl").write_text("".join(json.dumps(r) + "\n" for r in old), encoding="utf-8")
self.assertEqual(self._prompt("s2"), ["BROADCAST Three: old board", "DIRECT Three to Two: old direct"])
self.assertEqual(float((self.root / "news-seen" / "s2").read_text(encoding="utf-8")), now - 5)
self._send("s1", "send", "security", "new")
rows = self._rows()
self.assertEqual(rows[:2], old)
for row in rows:
self.assertIsInstance(row["ts"], float)
self.assertIsInstance(row["from"], str)
self.assertIsInstance(row["to"], list)
self.assertIsInstance(row["text"], str)
self.assertEqual(rows[2]["class"], "security")
def test_ts_strictly_increases_under_the_lock(self):
self.root.mkdir()
row = {"ts": time.time() + 1000, "from": "Three", "to": [], "text": "clock ahead"}
(self.root / "news.jsonl").write_text(json.dumps(row) + "\n", encoding="utf-8")
self._send("s1", "post", "first")
self._send("s1", "post", "second")
stamps = [r["ts"] for r in self._rows()]
self.assertGreater(stamps[1], stamps[0])
self.assertGreater(stamps[2], stamps[1])
def test_mayday_sets_silence_and_only_its_sender_clears_it(self):
self.assertEqual(self._send("s1", "clear"), ["no radio silence active"])
self.assertEqual(self._send("s1", "send", "mayday", "disk full")[0], "sent mayday to all")
silence = self.root / "radio-silence.json"
state = json.loads(silence.read_text(encoding="utf-8"))
self.assertEqual(sorted(state), ["by", "since", "text", "until"])
self.assertEqual((state["by"], state["text"], state["until"] - state["since"]), ("One", "disk full", 7200))
self.assertEqual(state["since"], self._rows()[-1]["ts"])
held = self._cli("s2", "clear")
self.assertEqual(held.returncode, 3)
self.assertIn("held by One", held.stdout)
self.assertTrue(silence.exists())
self.assertEqual(self._send("s1", "clear"), ["cleared radio silence by One"])
self.assertFalse(silence.exists())
self._send("s1", "send", "mayday", "again")
self.assertEqual(self._send("s2", "clear", "--force"), ["cleared radio silence by One, forced by Two"])
self.assertFalse(silence.exists())
def test_a_wake_names_every_addressee_without_a_live_listener(self):
self._pidfile("s2").parent.mkdir(parents=True)
self._pidfile("s2").write_text(str(os.getpid()), encoding="utf-8")
self.assertEqual(self._send("s1", "send", "mayday", "stop"), ["sent mayday to all", "wake: SendMessage to Three"])
self.assertEqual(
self._send("s1", "send", "direct", "answer", "--to", "Two", "--to", "Three", "--wake"),
["sent direct to Two, Three", "wake: SendMessage to Three"],
)
dead = subprocess.Popen([sys.executable, "-c", "pass"])
dead.wait()
self._pidfile("s2").write_text(str(dead.pid), encoding="utf-8")
self.assertEqual(
self._send("s1", "send", "direct", "answer", "--to", "Two", "--wake"),
["sent direct to Two", "wake: SendMessage to Two"],
)
self.assertEqual(self._send("s1", "send", "direct", "no wake", "--to", "Three"), ["sent direct to Three"])
self.assertEqual([r.get("wake") for r in self._rows()], [None, True, True, None])
def test_prompt_hook_puts_urgent_first_and_repeats_the_silence(self):
self._send("s1", "post", "board")
self._send("s1", "post", "for two", "--to", "Two")
self._send("s1", "send", "security", "sec")
self._send("s1", "send", "pan-pan", "pan")
self._send("s1", "send", "mayday", "may")
lines = self._prompt("s2")
self.assertRegex(lines[0], r"^MAYDAY ACTIVE since \d\d:\d\d by One: may; no new heavy jobs until radio\.py clear$")
self.assertEqual(
lines[1:],
["MAYDAY One: may", "PAN-PAN One: pan", "SECURITY One: sec", "BROADCAST One: board", "DIRECT One to Two: for two"],
)
self.assertEqual(len(self._prompt("s2")), 1)
self.assertNotIn("DIRECT One to Two: for two", self._prompt("s3"))
self.assertEqual(len(self._prompt("s1")), 1)
def test_prompt_hook_shows_ten_and_counts_the_rest(self):
self._send("s1", "send", "security", "early warning")
for n in range(12):
self._send("s1", "post", f"item {n}")
lines = self._prompt("s2")
self.assertEqual(lines[0], "SECURITY One: early warning")
self.assertEqual(lines[1:-1], [f"BROADCAST One: item {n}" for n in range(3, 12)])
self.assertEqual(lines[-1], "RADIO 3 more items: radio.py show --all")
self.assertEqual(self._prompt("s2"), [])
def test_tool_hook_delivers_urgent_rows_once_and_the_prompt_skips_them(self):
self._send("s1", "post", "board")
self.assertEqual(self._tool("s2"), [])
self._send("s1", "send", "pan-pan", "pan")
self.assertEqual(self._tool("s2"), ["PAN-PAN One: pan"])
self.assertEqual(self._tool("s2"), [])
self.assertEqual(self._prompt("s2"), ["BROADCAST One: board"])
self.assertEqual(self._tool("s3"), ["PAN-PAN One: pan"])
self.assertEqual(self._tool("s1"), [])
def test_tool_hook_skips_parsing_while_the_store_is_unchanged(self):
self._send("s1", "post", "x")
self.assertEqual(self._tool("s2"), [])
news = self.root / "news.jsonl"
before = news.stat()
row = {"ts": time.time(), "from": "Three", "to": [], "text": "hidden", "class": "security"}
with news.open("a", encoding="utf-8") as handle:
handle.write(json.dumps(row) + "\n")
os.utime(news, ns=(before.st_atime_ns, before.st_mtime_ns))
self.assertEqual(self._tool("s2"), [])
os.utime(news, ns=(before.st_atime_ns, before.st_mtime_ns + 10**9))
self.assertEqual(self._tool("s2"), ["SECURITY Three: hidden"])
def test_hooks_skip_subagents_and_never_fail(self):
self._send("s1", "send", "pan-pan", "pan")
self.assertEqual(self._hook("hook", "s2", agent_id="a1"), "")
self.assertEqual(self._hook("tool", "s2", agent_id="a1"), "")
listen = subprocess.run(
[sys.executable, "-c", LISTENER, str(RADIO)],
input=json.dumps({"session_id": "s2", "agent_id": "a1"}),
capture_output=True, text=True, env=self._env(None), check=False, timeout=30,
)
self.assertEqual((listen.returncode, listen.stderr), (0, ""))
self.assertFalse(self._pidfile("s2").exists())
self.assertFalse((self.root / "news-seen" / "s2").exists())
for command in ("hook", "tool"):
for stdin in ("not json", "{}", "[]"):
with self.subTest(command=command, stdin=stdin):
result = self._exec(None, stdin, (command,))
self.assertEqual((result.returncode, result.stdout), (0, ""))
def test_the_first_hook_notifies_the_operator_once(self):
log = self.root.parent / "notified.log"
script = "import sys; open(sys.argv[1], 'a').write(sys.argv[2] + '\\n')"
self.config.write_text(json.dumps({"notify": [sys.executable, "-c", script, str(log)]}), encoding="utf-8")
self._send("s1", "send", "mayday", "smoke")
self._prompt("s2")
self._prompt("s3")
self.assertEqual(log.read_text(encoding="utf-8"), "Agents radio MAYDAY by One: smoke\n")
def test_listener_wakes_on_mayday(self):
proc = self._listen("s2")
self.assertEqual(
self._send("s1", "send", "mayday", "host down"), ["sent mayday to all", "wake: SendMessage to Three"]
)
code, err = self._finish(proc)
self.assertEqual(code, 2)
self.assertEqual([strip(line) for line in err.splitlines()], ["MAYDAY One: host down"])
self.assertEqual(self._tool("s2"), [])
self.assertEqual(len(self._prompt("s2")), 1)
def test_listener_wakes_only_on_a_direct_wake(self):
proc = self._listen("s2")
self._send("s1", "post", "board")
self._send("s1", "post", "plain", "--to", "Two")
self._send("s1", "send", "pan-pan", "pan")
self._send("s1", "send", "direct", "elsewhere", "--to", "Three", "--wake")
time.sleep(0.5)
self.assertIsNone(proc.poll())
self.assertEqual(self._send("s1", "send", "direct", "answer me", "--to", "Two", "--wake"), ["sent direct to Two"])
code, err = self._finish(proc)
self.assertEqual(code, 2)
self.assertEqual([strip(line) for line in err.splitlines()], ["PAN-PAN One: pan", "DIRECT One to Two: answer me"])
self.assertEqual(self._prompt("s2"), ["BROADCAST One: board", "DIRECT One to Two: plain"])
def test_listener_yields_to_a_newer_one(self):
first = self._listen("s2")
second = self._listen("s2")
self.assertEqual(self._finish(first), (0, ""))
self.assertIsNone(second.poll())
self._pidfile("s2").write_text("0", encoding="utf-8")
self.assertEqual(self._finish(second), (0, ""))
if __name__ == "__main__":
unittest.main()
+34
View File
@@ -105,6 +105,19 @@ class TestWatchCommands(unittest.TestCase):
self._stage("warn")
self.assertEqual(self._run("block", stdin=self._bash("make compose-deploy")).returncode, 0)
def test_block_heavy_jobs_while_a_mayday_holds_radio_silence(self):
self._stage("ok")
self.root.mkdir(exist_ok=True)
silence = self.root / "radio-silence.json"
now = time.time()
silence.write_text(json.dumps({"since": now, "until": now + 60, "by": "One", "text": "fire"}), encoding="utf-8")
blocked = self._run("block", stdin=self._bash("make compose-deploy"))
self.assertEqual(blocked.returncode, 2)
self.assertIn("mayday by One", blocked.stderr)
self.assertEqual(self._run("block", stdin=self._bash("make swarm-down")).returncode, 0)
silence.write_text(json.dumps({"since": now - 120, "until": now - 60, "by": "One", "text": "fire"}), encoding="utf-8")
self.assertEqual(self._run("block", stdin=self._bash("make compose-deploy")).returncode, 0)
def test_block_ignores_heavy_words_inside_quotes_and_heredocs(self):
self.cfg["heavy_commands"] = [r"\bmake\s+(-C\s+\S+\s+)?(compose-deploy|test|swarm-\S+)(\s|$)"]
self._write_config()
@@ -152,6 +165,27 @@ class TestWatchCommands(unittest.TestCase):
self.assertEqual(hook("s1"), "")
self.assertIn("RESOURCE WARN", hook("s2"))
def test_watch_keeps_each_new_queue_history_row_once(self):
self.cfg["thresholds_gb"] = {"warn": 0, "stop": 0, "critical": 0}
self._write_config()
self.root.mkdir()
source = self.root / "queue-history.jsonl"
kept = self.base / ".local" / "state" / "agents-council" / "queue-history.jsonl"
kept.parent.mkdir(parents=True)
kept.write_text('{"id": "old"}\n', encoding="utf-8")
source.write_text('{"id": "old"}\n{"id": "a"}\n{broken\n', encoding="utf-8")
payload = json.dumps({"session_id": "s1", "hook_event_name": "PostToolUse"})
def kept_ids() -> list[str]:
self.assertEqual(self._run("watch", stdin=payload).returncode, 0)
return [json.loads(line)["id"] for line in kept.read_text(encoding="utf-8").splitlines()]
self.assertEqual(kept_ids(), ["old", "a"])
self.assertEqual(kept_ids(), ["old", "a"])
with source.open("a", encoding="utf-8") as handle:
handle.write('{"id": "b"}\n')
self.assertEqual(kept_ids(), ["old", "a", "b"])
def test_cache_trim_evicts_least_recently_used_first(self):
old, new = self.cache / "old", self.cache / "new"
old.write_bytes(b"x" * 4096)