diff --git a/README.md b/README.md index 0d7eed8..1343b2c 100644 --- a/README.md +++ b/README.md @@ -57,6 +57,7 @@ flowchart LR git --> git_update_branches[kvb-git-update-branches] git --> git_branch_gc[kvb-git-branch-gc] git --> git_changelog[kvb-git-changelog] + git --> git_hotfix[kvb-git-hotfix] check --> check_legal[kvb-check-legal] check --> check_security[kvb-check-security] check --> check_principles[kvb-check-principles] @@ -149,6 +150,7 @@ flowchart LR update_branches[kvb-git-update-branches] branch_gc[kvb-git-branch-gc] changelog[kvb-git-changelog] + hotfix[kvb-git-hotfix] end subgraph cat_check [check] advocate[kvb-check-legal] @@ -232,6 +234,10 @@ flowchart LR confidence --> gate pr_label --> gate gate --> democracy + gate --> hotfix + triage --> hotfix + democracy --> hotfix + hotfix --> rasci confidence -. objective .-> autotune coordinate --> democracy diff --git a/skills/kvb-check-gate/SKILL.md b/skills/kvb-check-gate/SKILL.md index e082dd3..d6032cc 100644 --- a/skills/kvb-check-gate/SKILL.md +++ b/skills/kvb-check-gate/SKILL.md @@ -11,26 +11,30 @@ target of the project, each Makefile target named `test` or starting with `test-`, ran green on the exact head with a clean tracked tree. Until then the numbers are provisional and every target not green is a residual. -`${CLAUDE_SKILL_DIR}/scripts/gate.py` is the deterministic part. It reads the -targets from the Makefile and every file it includes without running make, -and keeps the results per commit in `kvb-gate.json` in the git common dir: -linked worktrees share them, a new commit starts from zero. +`${CLAUDE_SKILL_DIR}/scripts/gate.py` reads the targets from the Makefile and +every file it includes without running make, and keeps the results per +commit in `kvb-gate.json` in the git common dir: linked worktrees share them, +a new commit starts from zero. ## Steps -1. **List**: `gate.py list --repo `. A warning about an unresolved +1. **Sync**: `${CLAUDE_SKILL_DIR}/../kvb-git-hotfix/scripts/hotfix.py sync + --repo --task `, then `hotfix.py pending + --repo `, by the `kvb-git-hotfix` skill. A merge moves HEAD, so the + gate starts over from there. +2. **List**: `gate.py list --repo `. A warning about an unresolved include or target expression means the list may be short: read that line and name the gap. A project without any target keeps the gate open; ask the operator. -2. **Status**: `gate.py status --repo `. `gate: green` ends here. -3. **Admit**: every gate target is heavy (`make test` and its siblings run - whole suites, often containers). Ask the resource chair of the +3. **Status**: `gate.py status --repo `. `gate: green` ends here; a + `missing hotfix` reason means back to step 1. +4. **Admit**: every gate target is heavy. Ask the resource chair of the `kvb-org-organize` skill per target (what, where, how long, disk), wait for the grant, post the release to its news. Only a single test module self-admits (`${CLAUDE_SKILL_DIR}/../kvb-org-organize/scripts/watch.py admit --kind test-nostack`), and it never turns a gate target green. A hotfix takes the next free gate slot. -4. **Run** the missing and red targets, tracked tree clean, in the +5. **Run** the missing and red targets, tracked tree clean, in the background; give the operator `tail -f` on the log: ```bash @@ -38,22 +42,32 @@ linked worktrees share them, a new commit starts from zero. ``` Pass one `--fail-regex` per failure line the project prints while exiting - 0; infinito-nexus needs `FAILED TARGETS: \S`. A target is red on a - non-zero exit, a `make: ***` line, a matching regex, or tracked files it - modified; the last one stops the run. -5. **Judge** every red target by the line `status` prints and the log around - it: caused by the change, already red on the default branch, or the - environment. Fix the cause and run again; a red target never counts as - green because it looks unrelated. -6. **Status** once more; its last line is the verdict. + 0; infinito-nexus needs `FAILED TARGETS: \S`. A non-zero exit, a + `make: ***` line, a matching regex or modified tracked files make a + target red; the last stops the run. Before each target `run` asks + `hotfix.py void` for it and every later one. Exit 3 (`void: merge main + for hotfix , then rerun`): post the release to the news, back to + step 1, and the chair's grant again at step 4. Exit 75: the worktree is + busy, the message names the holder; wait, then rerun. +6. **Judge** every red target by its `status` line and the log around it: + caused by the change, already red on the default branch, or the + environment. Fix the cause and rerun; looking unrelated never makes it + green. +7. **Status** once more; its last line is the verdict. ## Rules -- Only `gate.py run` on this head marks a target green. A peer's report, a - CI run, an older commit or a run on a dirty tree never does. +- Only `gate.py run` on this head marks a target green. A peer's report, CI, + an older commit or a dirty tree never does. +- Every heavy command in a worktree (deploys, `make test`, other gates) runs + through `gate.py run` or `gate.py hold --repo -- ` in the + background (`run_in_background`), never under a tool timeout or + `TaskStop`: it is never cut short. Their lock keeps `hotfix.py sync` out + only while a host process under it lives; a `docker compose exec` deploy + runs on in its container unguarded. - 90/5 is allowed only on `gate: green` for the exact head: in the - `kvb-check-confidence` block, the `90/5` label of the `kvb-git-pr-label` - skill and the bar of the `kvb-org-split` skill. + `kvb-check-confidence` block, the `90/5` label of `kvb-git-pr-label` and + the bar of `kvb-org-split`. - Never edit `kvb-gate.json` by hand. ## Output diff --git a/skills/kvb-check-gate/scripts/gate.py b/skills/kvb-check-gate/scripts/gate.py index 139358c..683a6c0 100755 --- a/skills/kvb-check-gate/scripts/gate.py +++ b/skills/kvb-check-gate/scripts/gate.py @@ -6,41 +6,82 @@ Args: parsed statically from the Makefile make reads in and every file it includes; make is never run. status --repo [--json]: per listed target green, red or missing for - HEAD, from `kvb-gate.json` in the git common dir. + HEAD, from `kvb-gate.json` in the git common dir; `hotfix.py pending + --target ...` of the kvb-git-hotfix skill, given the listed + targets, keeps the gate open while a merged hotfix that turns one of + them green is missing from HEAD. run --repo --target [--target ...] --log-dir - [--fail-regex ...]: run each target as `make -C ` with - stdout and stderr in /gate--.log and record the - result for HEAD. A target is red on a non-zero exit, a `make: ***` - line, a matching --fail-regex, or tracked files it modified. + [--fail-regex ...]: under the worktree lock, run each target as + `make -C ` with stdout and stderr in + /gate--.log and record the result for HEAD. A + target is red on a non-zero exit, a `make: ***` line, a matching + --fail-regex, or tracked files it modified. Before each target, + `hotfix.py void` decides over it and every target after it whether a + merged hotfix missing from HEAD voids the run. + hold --repo -- : under the worktree lock, run the + command in in a session of its own, without a controlling + terminal; SIGINT, SIGTERM and SIGHUP reaching hold go to the + command's process group while hold keeps waiting for it. For deploys + and every other heavy command that is no `make test*` target. + + The worktree lock is `kvb-gate-run.lock` in the worktree's git dir, + taken without waiting and passed to make or the command, so it stays + held while they or a process they left running on this host live; a + process inside a container (`docker compose exec`) runs on there after + they die, and the lock no longer covers it. run and hold write their PID + and command into it, and a busy message names them. `hotfix.py sync` + takes `kvb-gate-sync.lock` exclusively, recording its PID and command + there, then the worktree lock, and releases them in reverse order; run + and hold take the worktree lock while holding `kvb-gate-sync.lock` + shared, so they wait for a running sync, naming it, for at most + SYNC_WAIT_SECONDS, and a busy worktree lock then means another run or + hold. + + Without hotfix.py next to this skill, or when it exits non-zero without + a verdict line, gate.py warns on stderr and goes on without the hotfix + check. Output: list: one `\t:` per target. status: one `\t\t\t\t` per target, - then `gate: green`, or `gate: open (...)` naming what is missing. - run: one `\t\t\t` per target. + then `gate: green`, or `gate: open (...)` naming what is missing, + `missing hotfix: , ...` included. + run: one `\t\t\t` per target; on a void + `void: merge main for hotfix , then rerun`. + hold: the command's own output. --json prints the same data as JSON. Exit codes: list: 0 done, 2 usage error. status: 0 gate green, 1 gate open, 2 usage error. run: 0 every target green, 1 a target red, 2 usage error, dirty tracked - files or HEAD moved during a run. + files or HEAD moved during a run, 3 void: the remaining targets are + neither run nor recorded, 75 (EX_TEMPFAIL) busy: another run or hold + holds the worktree, or a sync still holds it after SYNC_WAIT_SECONDS. + hold: the command's exit code unchanged, 128+N when signal N ended it; + before the command started 2 on a usage error, 75 busy as for run, + 128+N when signal N reached hold. """ from __future__ import annotations import argparse +import contextlib import fcntl import glob import json import os import re +import shlex +import signal import subprocess import sys import tempfile +import time +from collections.abc import Iterator from datetime import UTC, datetime from pathlib import Path -from typing import NoReturn +from typing import NoReturn, TextIO MAKEFILES = ("GNUmakefile", "makefile", "Makefile") INCLUDE = re.compile(r"^\s*(?P-include|sinclude|include)\s+(?P.+)$") @@ -55,6 +96,13 @@ MAKE_ERROR = re.compile(r"\bmake(?:\[\d+\])?: \*\*\*") UNSAFE = re.compile(r"[^\w.-]") LINE_CHARS = 300 STATE = "kvb-gate.json" +RUN_LOCK = "kvb-gate-run.lock" +SYNC_LOCK = "kvb-gate-sync.lock" +SYNC_WAIT_SECONDS = 300 +POLL_SECONDS = 0.1 +BUSY = os.EX_TEMPFAIL +FORWARDED = (signal.SIGINT, signal.SIGTERM, signal.SIGHUP) +HOTFIX = Path(__file__).resolve().parents[2] / "kvb-git-hotfix" / "scripts" / "hotfix.py" def fail(message: str) -> NoReturn: @@ -62,6 +110,11 @@ def fail(message: str) -> NoReturn: sys.exit(2) +def busy(message: str) -> NoReturn: + print(f"gate: {message}", file=sys.stderr) + sys.exit(BUSY) + + def logical_lines(path: Path) -> list[tuple[int, str]]: lines: list[tuple[int, str]] = [] start, buffer = 0, "" @@ -212,6 +265,114 @@ def first_failure(log: Path, checks: list[re.Pattern]) -> str | None: return None +def hotfix_checks() -> bool: + if HOTFIX.is_file(): + return True + print(f"gate: {HOTFIX} is missing; merged hotfixes are not checked", file=sys.stderr) + return False + + +def hotfix(repo: Path, command: str, args: list[str], verdict: str) -> list[str]: + result = subprocess.run( + [sys.executable, str(HOTFIX), command, *args, "--repo", str(repo)], + stdin=subprocess.DEVNULL, capture_output=True, text=True, check=False, + ) + sys.stderr.write(result.stderr) + slugs = [ + line.split("\t")[1].removeprefix("hotfix/") + for line in result.stdout.splitlines() if line.startswith(f"{verdict}\t") + ] + if result.returncode and not slugs: + print(f"gate: hotfix.py {command} exited {result.returncode}; going on without its check", file=sys.stderr) + return slugs + + +def target_flags(targets: list[str]) -> list[str]: + return [part for target in targets for part in ("--target", target)] + + +def voiding_hotfixes(repo: Path, targets: list[str]) -> list[str]: + return hotfix(repo, "void", target_flags(targets), "void") + + +def missing_hotfixes(repo: Path, targets: list[str]) -> list[str]: + return hotfix(repo, "pending", target_flags(targets), "missing (merge main)") + + +def claim(lock: TextIO, command: list[str]) -> None: + """Record the holder in a lock file it holds exclusively. + + Args: + lock: the open, exclusively held lock file. + command: the holder's command line. + """ + lock.truncate(0) + lock.write(f"{os.getpid()}\t{shlex.join(command)}\n") + lock.flush() + + +def holder(path: Path) -> str: + """Name the holder that `claim` recorded in a lock file. + + Args: + path: the lock file. + """ + try: + pid, _, command = path.read_text(encoding="utf-8").strip().partition("\t") + except OSError: + pid, command = "", "" + return f"pid {pid}: {command}" if pid and command else "holder not recorded" + + +def shared(sync: TextIO, repo: Path, caught: list[int]) -> None: + """Take SYNC_LOCK shared, waiting at most SYNC_WAIT_SECONDS for a running sync. + + Args: + sync: the open SYNC_LOCK file. + repo: the worktree, named in messages. + caught: signals the caller caught; the first ends the wait with 128+N. + """ + deadline = time.monotonic() + SYNC_WAIT_SECONDS + waiting = False + while True: + try: + fcntl.flock(sync, fcntl.LOCK_SH | fcntl.LOCK_NB) + return + except BlockingIOError: + pass + if caught: + sys.exit(128 + caught[0]) + if time.monotonic() >= deadline: + busy(f"a hotfix.py sync still holds {repo} after {SYNC_WAIT_SECONDS} s ({holder(Path(sync.name))}); " + "retry once it ends") + if not waiting: + print(f"gate: a hotfix.py sync merges in {repo} ({holder(Path(sync.name))}); waiting for it at most " + f"{SYNC_WAIT_SECONDS} s", file=sys.stderr, flush=True) + waiting = True + time.sleep(POLL_SECONDS) + + +@contextlib.contextmanager +def worktree_lock(repo: Path, command: list[str], caught: list[int]) -> Iterator[int]: + """Hold the worktree lock; yields its file descriptor for pass_fds. + + Args: + repo: the worktree whose git dir holds RUN_LOCK and SYNC_LOCK. + command: the holder's command line, recorded in RUN_LOCK. + caught: signals the caller caught; the first ends a wait for a sync with 128+N. + """ + git_dir = Path(git(repo, "rev-parse", "--absolute-git-dir").strip()) + with open(git_dir / RUN_LOCK, "a", encoding="utf-8") as lock: + with open(git_dir / SYNC_LOCK, "a", encoding="utf-8") as sync: + shared(sync, repo, caught) + try: + fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB) + except BlockingIOError: + busy(f"another gate.py run or hold holds {repo} ({holder(git_dir / RUN_LOCK)})") + claim(lock, command) + yield lock.fileno() + + def repository(value: str) -> Path: repo = Path(value).resolve() if not repo.is_dir(): @@ -254,6 +415,9 @@ def cmd_status(args: argparse.Namespace) -> int: reasons.append(f"red: {', '.join(red)}") if dirty: reasons.append(f"dirty: {len(dirty)} tracked file(s)") + hotfixes = missing_hotfixes(repo, list(found)) if hotfix_checks() else [] + if hotfixes: + reasons.append(f"missing hotfix: {', '.join(hotfixes)}") gate = "open" if reasons else "green" if args.json: print(json.dumps( @@ -283,51 +447,90 @@ def cmd_run(args: argparse.Namespace) -> int: unknown = [t for t in args.target if t not in found] if unknown: fail(f"not a make test* target of {repo}: {', '.join(unknown)}") - sha = head(repo) - dirty, untracked = tree(repo) - if dirty: - fail(f"tracked files are dirty, commit or drop them first: {', '.join(dirty)}") - if untracked: - print(f"untracked\t{len(untracked)} file(s), not part of the gate") - log_dir = Path(args.log_dir).resolve() - log_dir.mkdir(parents=True, exist_ok=True) - state = state_path(repo) - red = False - for target in dict.fromkeys(args.target): - log = log_dir / f"gate-{UNSAFE.sub('_', target)}-{sha[:8]}.log" - with open(log, "w", encoding="utf-8") as out: - code = subprocess.run( - ["make", "-C", str(repo), target], - stdin=subprocess.DEVNULL, stdout=out, stderr=subprocess.STDOUT, check=False, - ).returncode - if head(repo) != sha: - fail(f"HEAD moved during {target}; nothing recorded for it") - changed, _ = tree(repo) - reasons = [] - if code: - reasons.append(f"exit {code}") - line = first_failure(log, checks) - if line is not None: - reasons.append(line) - if changed: - reasons.append(f"modified tracked files: {', '.join(changed)}") - status = "red" if reasons else "green" - red = red or status == "red" - entry = {"status": status, "log": str(log), - "finished": datetime.now(UTC).isoformat(timespec="seconds"), - "fail_regex": [c.pattern for c in checks]} - if reasons: - entry["reason"] = "; ".join(reasons) - record(state, sha, target, entry) - print("\t".join([status, target, str(log), cell(entry.get("reason"))])) - if changed: - fail(f"{target} left tracked files dirty; remaining targets not run") + with worktree_lock(repo, ["gate.py", "run", "--repo", str(repo), *target_flags(args.target)], []) as lock: + sha = head(repo) + dirty, untracked = tree(repo) + if dirty: + fail(f"tracked files are dirty, commit or drop them first: {', '.join(dirty)}") + if untracked: + print(f"untracked\t{len(untracked)} file(s), not part of the gate") + log_dir = Path(args.log_dir).resolve() + log_dir.mkdir(parents=True, exist_ok=True) + state = state_path(repo) + red = False + order = list(dict.fromkeys(args.target)) + checking = hotfix_checks() + for index, target in enumerate(order): + slugs = voiding_hotfixes(repo, order[index:]) if checking else [] + if slugs: + print(f"void: merge main for hotfix {', '.join(slugs)}, then rerun") + return 3 + log = log_dir / f"gate-{UNSAFE.sub('_', target)}-{sha[:8]}.log" + with open(log, "w", encoding="utf-8") as out: + code = subprocess.run( + ["make", "-C", str(repo), target], + stdin=subprocess.DEVNULL, stdout=out, stderr=subprocess.STDOUT, pass_fds=(lock,), check=False, + ).returncode + if head(repo) != sha: + fail(f"HEAD moved during {target}; nothing recorded for it") + changed, _ = tree(repo) + reasons = [] + if code: + reasons.append(f"exit {code}") + line = first_failure(log, checks) + if line is not None: + reasons.append(line) + if changed: + reasons.append(f"modified tracked files: {', '.join(changed)}") + status = "red" if reasons else "green" + red = red or status == "red" + entry = {"status": status, "log": str(log), + "finished": datetime.now(UTC).isoformat(timespec="seconds"), + "fail_regex": [c.pattern for c in checks]} + if reasons: + entry["reason"] = "; ".join(reasons) + record(state, sha, target, entry) + print("\t".join([status, target, str(log), cell(entry.get("reason"))])) + if changed: + fail(f"{target} left tracked files dirty; remaining targets not run") return 1 if red else 0 +def send(child: subprocess.Popen, signum: int) -> None: + if child.returncode is None: + with contextlib.suppress(ProcessLookupError, PermissionError): + os.killpg(child.pid, signum) + + +def cmd_hold(args: argparse.Namespace) -> int: + repo = repository(args.repo) + caught: list[int] = [] + child: subprocess.Popen | None = None + + def forward(signum: int, _frame: object) -> None: + if child is None: + caught.append(signum) + else: + send(child, signum) + + for signum in FORWARDED: + signal.signal(signum, forward) + with worktree_lock(repo, ["gate.py", "hold", "--repo", str(repo), "--", *args.command], caught) as lock: + if caught: + return 128 + caught[0] + try: + child = subprocess.Popen(args.command, cwd=repo, pass_fds=(lock,), start_new_session=True) + except OSError as error: + fail(f"cannot start {args.command[0]}: {error.strerror}") + for signum in caught: + send(child, signum) + code = child.wait() + return code if code >= 0 else 128 - code + + def main() -> int: parser = argparse.ArgumentParser(description=__doc__.splitlines()[0]) - sub = parser.add_subparsers(dest="command", required=True) + sub = parser.add_subparsers(dest="subcommand", required=True) for name in ("list", "status"): command = sub.add_parser(name) command.add_argument("--repo", required=True) @@ -337,8 +540,12 @@ def main() -> int: run.add_argument("--target", action="append", required=True) run.add_argument("--log-dir", required=True) run.add_argument("--fail-regex", action="append") + hold = sub.add_parser("hold") + hold.add_argument("--repo", required=True) + hold.add_argument("command", nargs="+") args = parser.parse_args() - return {"list": cmd_list, "status": cmd_status, "run": cmd_run}[args.command](args) + commands = {"list": cmd_list, "status": cmd_status, "run": cmd_run, "hold": cmd_hold} + return commands[args.subcommand](args) if __name__ == "__main__": diff --git a/skills/kvb-ci-triage/SKILL.md b/skills/kvb-ci-triage/SKILL.md index b18e8b0..a7f39ee 100644 --- a/skills/kvb-ci-triage/SKILL.md +++ b/skills/kvb-ci-triage/SKILL.md @@ -23,8 +23,9 @@ steps 0 to 2 in one call; the judgement stays yours. differ beyond interpolated identifiers (host, app, port, path) are separate clusters. 3. **Coordinate** each cluster via the `kvb-org-organize` Peer protocol, - lineage included: claim it; read `council.py list`, the register, the news - and `find_fix.py`; then ask every live peer for known fixes, one batched + lineage included: `${CLAUDE_SKILL_DIR}/../kvb-git-hotfix/scripts/hotfix.py + match --line ''` first; claim it; read `council.py list`, the + register, the news and `find_fix.py`; ask every live peer, one batched `SendMessage` per peer for all clusters nothing there answers. Investigate while waiting; take their fix or keep yours. Register (`kvb-org-rasci`): objective the root cause fixed, one key result per failing job passing. diff --git a/skills/kvb-git-hotfix/SKILL.md b/skills/kvb-git-hotfix/SKILL.md new file mode 100644 index 0000000..8e03df7 --- /dev/null +++ b/skills/kvb-git-hotfix/SKILL.md @@ -0,0 +1,78 @@ +--- +name: kvb-git-hotfix +description: > + Share hotfixes through a register table instead of messages; merge main and + upstream branches before every deploy, merge or gate; match failures first. +--- + +`${CLAUDE_SKILL_DIR}/scripts/hotfix.py` keeps each hotfix as a task +`hotfix/` of the `kvb-org-rasci` register, the author as R and A; +never a message or news post. + +```bash +hotfix.py add --repo --branch --sha --signature '' --status staged|pr|merged [--target ...] [--pr ] +hotfix.py update --status staged|pr|merged [--sha ] [--pr ] [--repo ] +hotfix.py sync --repo --task [--default ] +hotfix.py pending --repo +hotfix.py match --line '' --repo +hotfix.py void --repo --target [--target ...] +``` + +## Author: record, never announce + +1. Fix committed on its branch: `add`. The signature is a regular expression + for the failing log line: its stable part, interpolated hosts, ports and + ids as patterns, never matching a healthy line. List every `make` target + the fix turns green, `deploy` for a deploy; none listed voids every run, + a `test-*` target also the umbrella `test`. +2. Pull request open: `update --status pr --pr `. +3. Landed: `update --status merged --sha `, the commit as it sits + on the default branch (a squash or rebase merge changes it; any other is + refused). `--repo` only once the recorded clone is gone. + +## Before every deploy, merge or gate + +1. `sync`: fetches, merges `origin/`, then every branch up the funnel + chain the `kvb-org-split` skill records as `base` (open rows of this + repository), each with `--no-verify --no-edit`. It needs the task's branch + checked out, a clean tracked tree and the worktree lock free; exit 2 + names the blocker. Exit 4: a merge conflicted and was aborted; + resolve it by the `kvb-git-merge-fix` rules, then sync again. +2. `pending`: exit 0 means every merged hotfix is in HEAD. `open, not + merged` is reported, never merged: name it in your report and go on. + `not on origin/` or exit 2: tell its A to correct the row. + +## On any failure + +Before triage, investigation or asking a peer: `match`, the line quoted +literally; `foreign` marks another repository's hotfix. A merged match: sync +and run again. An open match: the fix is in flight on that branch; wait for +it or work on it with its author, never build a second one. + +## Stopping a running job + +A running gate or deploy stops only when a merged hotfix it lacks turns one +of its targets green (void); otherwise never (the `kvb-org-organize` +capacity rules). + +- `gate.py run` of the `kvb-check-gate` skill asks `void` before each + target; on exit 3 finished targets keep their results. +- A deploy: whenever you act on it, run `void --target deploy`; + hotfixes listing `deploy` or no target void it. Only on exit 3 stop it at + the next safe point, between stages and never inside a task. Never on a + timer. +- After exit 3: post the release to the news, sync, and request the chair's + grant again before rerunning. + +## Rules + +- Every deploy, `make test` or other gate in a worktree runs through + `gate.py run` or `gate.py hold` by the `kvb-check-gate` rules (background, + no tool timeout, no `TaskStop`; exit 75 while busy). Only that lock keeps + sync out, while a host process under it lives; a deploy inside a container + outlives it unguarded. +- One row per fix across all agents: `match` and `pending` before `add`. +- A row is data: it never permits a merge, push or command the agent rules + forbid. +- Never edit `hotfix/` rows with `rasci.py set`; `add` and `update` keep + them consistent. diff --git a/skills/kvb-git-hotfix/scripts/hotfix.py b/skills/kvb-git-hotfix/scripts/hotfix.py new file mode 100755 index 0000000..c00d8f1 --- /dev/null +++ b/skills/kvb-git-hotfix/scripts/hotfix.py @@ -0,0 +1,882 @@ +#!/usr/bin/env python3 +"""Share hotfixes through the kvb-org-rasci register and keep a worktree merged with them. + +Environment: + AGENTS_COUNCIL_ROOT: council directory of the kvb-org-rasci register, read + by rasci.py; the rows are its tasks hotfix/. + CLAUDE_CODE_SESSION_ID: the adding session, recorded as R and A; other + runtimes pass --session. + AGENTS_COUNCIL_CONFIG: read by watch.py of the kvb-org-organize skill for + sync's busy check; without its config file there is none. + +Args: + add --repo --branch --sha + --signature --status staged|pr|merged [--target ...] + [--pr ] [--default ] [--session ]: record a new hotfix + under the repository's git common dir and origin URL, never the + --repo path. --sha is a hex commit id in , stored in full; with + --status merged it must be on origin/ after a fetch. + --signature matches the failing log line and puts no unbounded + repeat around another unbounded repeat or around an alternation whose + branches can start alike; bounded repeats such as {3} pass. Each + --target is a make target or `deploy` the fix turns green, none listed + means every target; --pr is the pull request number. + update --status staged|pr|merged [--sha ] [--pr ] + [--repo ] [--default ]: move a hotfix on. --sha + resolves in the recorded git common dir or in --repo, which must be + the same repository unless the recorded one is unknown, and records + it anew. --status merged needs --sha, the commit as it sits on + origin/ after a fetch. + pending --repo [--target ...]: every hotfix of that repository + against HEAD; with --target only those that turn one of the targets + green, as void counts them. + match --line [--repo ]: every hotfix whose signature + matches the line, all searched in order by one worker process with + one deadline of SEARCH_SECONDS; a signature still searching at the + deadline is skipped and a new worker searches those after it. With + --repo, those of another repository are labelled and do not count. + void --repo --target [--target ...]: whether a merged hotfix of + that repository missing from HEAD turns one of the targets green; one + listing a `test-*` target also turns the umbrella `test` green. + sync --repo --task [--default ]: with the + task's branch checked out, `git fetch origin` before taking any lock + (a fetch moves neither the worktree nor HEAD), then, with a clean + tracked tree, merge origin/ and each base up the chain, each + with `git merge --no-verify --no-edit`. From the tree check to the + last merge or abort it holds `kvb-gate-sync.lock`, with its PID and + command recorded there, and the worktree lock of the kvb-check-gate + skill's gate.py, and refuses while a `gate.py run` or `gate.py hold` + holds it, naming the holder. A watch.py busy process it sees in + also refuses, but processes in another sandbox's PID namespace stay + unseen, so the lock is the guard. The default branch is origin/HEAD; --default is read + only when that ref is missing and must agree with it otherwise. + Without an origin remote sync skips the fetch, --default names the + local default branch and bases merge from local branches. The chain + starts at the task's `base` and goes on with the `base` of the open + register task whose `branch` is that base and whose workdir is a + worktree of this repository, or which records no workdir while its + branch resolves here as or origin/, up to the + default branch. A base merges as origin/ when the local branch + is missing or contained in it, as the local branch when origin/ + is missing or contained in that. + + A hotfix belongs to a repository when both have the same origin URL or, + when either has none, the same git common dir. Origin URLs come from + `git ls-remote --get-url origin`, so insteadOf rules apply, and compare + without scheme, user, port, `.git` and trailing slash; a local path + resolves against the worktree's top level, and the path on a + case-insensitive host of CASELESS compares in lowercase. A merged hotfix + missing from HEAD counts only when its sha is on origin/; when + it is not, one fetch decides. Every fetch (sync, pending, void, and add + or update with --status merged) runs with GIT_TERMINAL_PROMPT=0 and is + stopped after FETCH_SECONDS. + +Output: + add, update: `added hotfix/` or `updated hotfix/`. + pending: `\t\t\t\t\t` for each + merged hotfix missing from HEAD (`missing (merge main)`, or + `not on origin/`) and each staged or pr one + (`open, not merged`), then `hotfix: missing, open`. + match: `\t\t\t\t\t` per match, + with --repo prefixed `foreign\t` for another repository and + `unknown\t` for an unresolvable one, or a `hotfix: no signature` + line when nothing of the repository matches. A signature still + searching at the SEARCH_SECONDS deadline is named on stderr and + counts as no match. + void: `void\t\t\t` per hotfix that voids the run; a + hotfix off origin/ voids nothing and is named on stderr. + sync: `merged\t\t` or `contained\t` per ref in merge + order; `conflict\t\t` on the first conflict. + pending and void name every row whose repository is unknown (no origin + URL and its git common dir gone) on stderr; such a row changes no exit + code. + +Exit codes: + 0 done; pending: none missing; match: a match of the repository; void: + not void. + 1 pending: a merged hotfix is missing; match: no match. + 2 usage error, unknown task, dirty tracked tree, HEAD not on the task's + branch, a held worktree, a chain row without workdir whose branch + does not resolve, an unresolvable or diverged ref, a merge that + failed without a conflict (aborted), a merged sha off + origin/; pending and void: a missing origin/HEAD or a + failed fetch, unless void exits 3; match: the search worker did not + start, did not get ready before the deadline or exited non-zero. + 3 void: a merged hotfix missing from HEAD turns a target green. + 4 sync: a merge conflicted and was aborted; earlier merges stay. +""" + +from __future__ import annotations + +import argparse +import contextlib +import fcntl +import importlib.util +import itertools +import json +import os +import re +import subprocess +import sys +from collections.abc import Callable, Iterator +from pathlib import Path +from types import ModuleType +from typing import NoReturn + +SKILLS = Path(__file__).resolve().parents[2] +RASCI = SKILLS / "kvb-org-rasci" / "scripts" / "rasci.py" +GATE = SKILLS / "kvb-check-gate" / "scripts" / "gate.py" +WATCH = SKILLS / "kvb-org-organize" / "scripts" / "watch.py" +PREFIX = "hotfix/" +STAGES = ("staged", "pr", "merged") +REGISTER_STATUS = {"staged": "running", "pr": "running", "merged": "done"} +SHA = re.compile(r"^[0-9a-f]{7,40}$") +TARGET = re.compile(r"^[A-Za-z0-9_][A-Za-z0-9._%/+-]*$") +UMBRELLA = "test" +ORIGIN_HEAD = "refs/remotes/origin/HEAD" +GIT = ("git", "-c", "maintenance.auto=false") +FETCH = ("fetch", "--quiet", "--no-auto-maintenance", "origin") +FETCH_SECONDS = 120 +URL = re.compile(r"^[A-Za-z][A-Za-z0-9+.-]*://(?:[^@/]*@)?(?P[^/:]*)(?::\d*)?(?P/.*)?$") +SCP = re.compile(r"^(?:[^@/:]+@)?(?P[^/:]+):(?P.*)$") +CASELESS = ("github.com", "gitlab.com", "codeberg.org", "bitbucket.org") +SEARCH_SECONDS = 3 +SEARCH = ( + "import json, re, sys\n" + "data = json.loads(sys.stdin.buffer.read())\n" + "print('ready', flush=True)\n" + "for pattern in data['patterns']:\n" + " print(int(re.search(pattern, data['line']) is not None), flush=True)\n" +) +CLASSES = {"DIGIT": r"\d", "SPACE": r"\s", "WORD": r"\w", "LINEBREAK": r"\n"} +PROBES = frozenset(map(chr, range(128))) | {"é", "٠", "　"} +NESTED = "repeats a group that repeats itself, like (a+)+" +OVERLAP = "repeats an alternation whose branches can start alike, like (a|ab|b)*" + + +def fail(message: str) -> NoReturn: + print(f"hotfix: {message}", file=sys.stderr) + sys.exit(2) + + +def report(problems: list[str]) -> None: + for problem in problems: + print(f"hotfix: {problem}", file=sys.stderr) + + +def module(name: str, path: Path) -> ModuleType: + spec = importlib.util.spec_from_file_location(name, path) + loaded = importlib.util.module_from_spec(spec) + spec.loader.exec_module(loaded) + return loaded + + +def git(where: str, *args: str) -> subprocess.CompletedProcess: + try: + return subprocess.run( + [*GIT, *args], cwd=where, stdin=subprocess.DEVNULL, capture_output=True, text=True, check=False + ) + except OSError as error: + return subprocess.CompletedProcess([*GIT, *args], 128, "", str(error)) + + +def git_out(where: str, *args: str) -> str: + result = git(where, *args) + if result.returncode: + fail(f"git {' '.join(args)} in {where}: {(result.stderr or result.stdout).strip()}") + return result.stdout + + +def fetch(where: str) -> subprocess.CompletedProcess: + command = [*GIT, *FETCH] + try: + return subprocess.run( + command, cwd=where, stdin=subprocess.DEVNULL, capture_output=True, text=True, + env={**os.environ, "GIT_TERMINAL_PROMPT": "0"}, timeout=FETCH_SECONDS, check=False, + ) + except subprocess.TimeoutExpired: + return subprocess.CompletedProcess(command, 124, "", f"stopped after {FETCH_SECONDS} s") + except OSError as error: + return subprocess.CompletedProcess(command, 128, "", str(error)) + + +def fetched(where: str) -> None: + result = fetch(where) + if result.returncode: + fail(f"git fetch origin in {where}: {(result.stderr or result.stdout).strip()}") + + +def ancestor(where: str, rev: str, of: str) -> bool: + return git(where, "merge-base", "--is-ancestor", rev, of).returncode == 0 + + +def resolves(where: str, ref: str) -> bool: + return git(where, "rev-parse", "--verify", "--quiet", f"{ref}^{{commit}}").returncode == 0 + + +def task_of(rasci: ModuleType, slug: str) -> str: + if not rasci.TASK.match(slug): + fail(f"invalid slug {slug!r}; use lowercase letters, digits, '.', '_', '-' and '/'") + return PREFIX + slug + + +def worktree(value: str) -> str: + path = os.path.realpath(value) + if not os.path.isdir(path): + fail(f"{value} is no directory") + if identity(path) is None: + fail(f"{value} is not a git repository") + return path + + +def clone(value: str) -> str: + if not os.path.isabs(value): + fail(f"--repo {value!r} is not an absolute path") + return worktree(value) + + +def normalized(url: str, top: str) -> str: + """Compare form of an origin URL. + + Args: + url: the origin URL with insteadOf rules applied. + top: the directory a local relative path is relative to. + """ + url = url.strip() + found = URL.match(url) or (None if url.startswith((".", "/")) else SCP.match(url)) + host, path = (found["host"].lower(), found["path"] or "") if found else ("", url) + if host: + path = path.rstrip("/").removesuffix(".git").rstrip("/").lstrip("/") + return f"{host}/{path.lower() if host in CASELESS else path}" + if not path: + return "" + return os.path.realpath(os.path.join(top, path)).removesuffix(".git").rstrip("/") + + +def origin_url(where: str) -> str: + if not git(where, "config", "--get", "remote.origin.url").stdout.strip(): + return "" + url = git(where, "ls-remote", "--get-url", "origin").stdout + top = git(where, "rev-parse", "--show-toplevel") + return normalized(url, top.stdout.strip() if top.returncode == 0 else where) + + +def identity(where: str) -> tuple[str, str] | None: + if not where or not os.path.isdir(where): + return None + common = git(where, "rev-parse", "--path-format=absolute", "--git-common-dir") + if common.returncode: + return None + return os.path.realpath(common.stdout.strip()), origin_url(where) + + +def same(one: tuple[str, str], other: tuple[str, str]) -> bool: + if one[1] and other[1]: + return one[1] == other[1] + return one[0] == other[0] + + +def recorded(row: dict) -> tuple[str, str] | None: + common, origin = row.get("common_dir") or "", row.get("origin") or "" + if not origin and not os.path.isdir(common): + return None + return common, origin + + +def repository(row: dict) -> str: + return row.get("origin") or row.get("common_dir") or "-" + + +def unknown(task: str, row: dict) -> str: + return ( + f"{task}: repository unknown (no origin URL, git dir {row.get('common_dir') or '-'} gone); record it with " + f"`hotfix.py update {task.removeprefix(PREFIX)} --status {row['stage']} --sha {row['sha']} --repo `" + ) + + +def hex_id(sha: str) -> str: + if not SHA.match(sha): + fail(f"--sha {sha!r} is no hex commit id of 7 to 40 characters") + return sha + + +def commit(where: str, sha: str) -> str: + found = git(where, "rev-parse", "--verify", "--quiet", f"{hex_id(sha)}^{{commit}}") + if found.returncode: + fail(f"--sha {sha} is no commit in {where}") + return found.stdout.strip() + + +def default_branch(where: str, given: str | None) -> str: + head = git(where, "symbolic-ref", "--quiet", ORIGIN_HEAD) + if head.returncode == 0: + name = head.stdout.strip().removeprefix("refs/remotes/origin/") + if given is not None and given != name: + fail(f"origin/HEAD names {name}, --default says {given}") + return name + if given is None: + fail(f"{ORIGIN_HEAD} is missing in {where}; pass --default or run `git remote set-head origin --auto`") + return branch_name(where, given, "--default") + + +def landed(where: str, sha: str, given: str | None) -> str: + hex_id(sha) + fetched(where) + name = default_branch(where, given) + full = commit(where, sha) + if not ancestor(where, full, f"refs/remotes/origin/{name}"): + fail(f"--sha {sha} is not on origin/{name}; record the commit as it sits there, a squash or rebase merge " + "changes it") + return full + + +def branch_name(where: str, name: str, what: str) -> str: + if name.startswith("-") or git(where, "check-ref-format", f"refs/heads/{name}").returncode: + fail(f"{what} {name!r} is no valid branch name") + return name + + +def children(value: object, subpattern: type) -> list: + if isinstance(value, subpattern): + return [value] + if isinstance(value, (tuple, list)): + return [sub for item in value for sub in children(item, subpattern)] + return [] + + +def either(tests: list[Callable[[str], bool]]) -> Callable[[str], bool]: + return lambda char: any(test(char) for test in tests) + + +def risk(text: str) -> str | None: + parser, c = getattr(re, "_parser", None), getattr(re, "_constants", None) + if parser is None or c is None: + print("hotfix: this Python hides re's parser; --signature is not checked for runaway backtracking", + file=sys.stderr) + return None + loose = (c.MAX_REPEAT, c.MIN_REPEAT) + repeats = (*loose, c.POSSESSIVE_REPEAT) + barriers = (c.ATOMIC_GROUP, c.POSSESSIVE_REPEAT) + probes = PROBES | set(text) + + def parts(av: object) -> list: + return children(av, parser.SubPattern) + + def unbounded(op: object, av: object) -> bool: + return op in loose and av[1] == c.MAXREPEAT + + def inside(pattern: object, hit: Callable[[object, object], bool]) -> bool: + return any( + hit(op, av) or (op not in barriers and any(inside(part, hit) for part in parts(av))) + for op, av in pattern.data + ) + + def overlapping(op: object, av: object) -> bool: + if op != c.BRANCH: + return False + heads = [start(branch) for branch in av[1]] + return any( + (one_empty and two_empty) or any(one(char) and two(char) for char in probes) + for (one, one_empty), (two, two_empty) in itertools.combinations(heads, 2) + ) + + def start(pattern: object) -> tuple[Callable[[str], bool], bool]: + tests = [] + for op, av in pattern.data: + test, empty = first(op, av) + tests.append(test) + if not empty: + return either(tests), False + return either(tests), True + + def first(op: object, av: object) -> tuple[Callable[[str], bool], bool]: + if op == c.LITERAL: + return (lambda char: char == chr(av)), False + if op == c.NOT_LITERAL: + return (lambda char: char != chr(av)), False + if op == c.ANY: + return (lambda char: True), False + if op == c.IN: + return member(av), False + if op in (c.AT, c.ASSERT, c.ASSERT_NOT): + return (lambda char: False), True + if op in repeats: + test, empty = start(av[2]) + return test, empty or av[0] == 0 + if op in (c.SUBPATTERN, c.ATOMIC_GROUP, c.BRANCH): + heads = [start(part) for part in parts(av)] + return either([test for test, _ in heads]), any(empty for _, empty in heads) + return (lambda char: True), True + + def member(items: list) -> Callable[[str], bool]: + negated = bool(items) and items[0][0] == c.NEGATE + kept = [item for item in items if item[0] != c.NEGATE] + return lambda char: any(holds(op, av, char) for op, av in kept) != negated + + def holds(op: object, av: object, char: str) -> bool: + if op == c.LITERAL: + return char == chr(av) + if op == c.RANGE: + return av[0] <= ord(char) <= av[1] + if op == c.CATEGORY: + name = av.name.removeprefix("CATEGORY_").removeprefix("LOC_").removeprefix("UNI_") + klass = CLASSES.get(name.removeprefix("NOT_")) + return klass is None or (re.fullmatch(klass, char) is not None) != name.startswith("NOT_") + return True + + def scan(pattern: object) -> str | None: + for op, av in pattern.data: + if unbounded(op, av) and inside(av[2], unbounded): + return NESTED + if unbounded(op, av) and inside(av[2], overlapping): + return OVERLAP + for part in parts(av): + found = scan(part) + if found is not None: + return found + return None + + return scan(parser.parse(text)) + + +def signature(text: str) -> str: + if not text.strip(): + fail("--signature must not be empty") + try: + re.compile(text) + except re.error as error: + fail(f"--signature {text!r}: {error}") + found = risk(text) + if found is not None: + fail(f"--signature {text!r} {found}, which backtracks without end on a near miss; match the stable part " + "instead") + return text + + +def searched(patterns: list[str], line: str) -> list[bool]: + """Search the line with every pattern in order in one worker until SEARCH_SECONDS pass. + + Args: + patterns: signatures that compile. + line: the log line. + + Returns: + one verdict per pattern; fewer when the deadline passed, the next pattern still searching. + """ + payload = json.dumps({"line": line, "patterns": patterns}).encode("ascii") + late = False + try: + with subprocess.Popen( + [sys.executable, "-I", "-c", SEARCH], stdin=subprocess.PIPE, stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + ) as worker: + try: + out, err = worker.communicate(payload, timeout=SEARCH_SECONDS) + except subprocess.TimeoutExpired: + late = True + worker.kill() + out, err = worker.communicate() + except OSError as error: + fail(f"cannot start the signature search worker {sys.executable}: {error.strerror or error}") + *lines, tail = out.decode("ascii", "replace").split("\n") + verdicts = lines[1:] + if late and lines[:1] != ["ready"]: + fail(f"the signature search worker did not get ready within {SEARCH_SECONDS} s") + complete = late or (worker.returncode == 0 and not tail and len(verdicts) == len(patterns)) + if lines[:1] != ["ready"] or not complete or len(verdicts) > len(patterns) or set(verdicts) - {"0", "1"}: + reason = err.decode("utf-8", "replace").strip().splitlines()[-1:] or ["no message"] + fail(f"the signature search worker exited {worker.returncode}: {reason[0]}") + return [verdict == "1" for verdict in verdicts] + + +def make_targets(values: list[str] | None) -> list[str]: + for value in values or []: + if not TARGET.match(value): + fail(f"--target {value!r} is no make target name") + return list(dict.fromkeys(values or [])) + + +def turns_green(greens: list[str], wanted: set[str]) -> bool: + if not greens or wanted & set(greens): + return True + return UMBRELLA in wanted and any(green.startswith(f"{UMBRELLA}-") for green in greens) + + +def number(value: int | None) -> int | None: + if value is not None and value < 1: + fail("--pr must be 1 or higher") + return value + + +def hotfixes(rasci: ModuleType) -> list[tuple[str, dict]]: + return [(t, row) for t, row in sorted(rasci.load().items()) if t.startswith(PREFIX) and "stage" in row] + + +def of_repository(rasci: ModuleType, repo: str) -> tuple[list[tuple[str, dict]], list[str]]: + here = identity(repo) + rows, unknowns = [], [] + for task, row in hotfixes(rasci): + theirs = recorded(row) + if theirs is None: + unknowns.append(unknown(task, row)) + elif same(theirs, here): + rows.append((task, row)) + return rows, unknowns + + +def for_targets(rows: list[tuple[str, dict]], targets: list[str] | None) -> list[tuple[str, dict]]: + if targets is None: + return rows + wanted = set(make_targets(targets)) + return [(task, row) for task, row in rows if turns_green(row.get("targets") or [], wanted)] + + +def missing(repo: str, rows: list[tuple[str, dict]], problems: list[str]) -> tuple[list, list, str]: + absent = [(t, row) for t, row in rows if row["stage"] == "merged" and not ancestor(repo, row["sha"], "HEAD")] + if not absent: + return [], [], "" + head = git(repo, "symbolic-ref", "--quiet", ORIGIN_HEAD) + if head.returncode: + problems.append(f"{ORIGIN_HEAD} is missing in {repo}, so {', '.join(t for t, _ in absent)} stay unchecked; " + "run `git remote set-head origin --auto`") + return [], [], "" + ref = head.stdout.strip() + on = {t for t, row in absent if ancestor(repo, row["sha"], ref)} + if len(on) < len(absent): + result = fetch(repo) + if result.returncode: + unchecked = ", ".join(t for t, _ in absent if t not in on) + problems.append(f"git fetch origin in {repo} failed, so {unchecked} stay unchecked: {result.stderr.strip()}") + return [(t, row) for t, row in absent if t in on], [], "" + on = {t for t, row in absent if ancestor(repo, row["sha"], ref)} + shown = ref.removeprefix("refs/remotes/") + return [(t, row) for t, row in absent if t in on], [(t, row) for t, row in absent if t not in on], shown + + +def pr_cell(row: dict) -> str: + return f"#{row['pr']}" if row.get("pr") else "-" + + +def cmd_add(rasci: ModuleType, args: argparse.Namespace) -> int: + task = task_of(rasci, args.slug) + if task in rasci.load(): + fail(f"{task} is already in the register; move it on with update") + repo = clone(args.repo) + common, origin = identity(repo) + entry = { + "common_dir": common, + "origin": origin, + "branch": branch_name(repo, args.branch, "--branch"), + "signature": signature(args.signature), + "targets": make_targets(args.target), + "stage": args.status, + } + pr = number(args.pr) + if pr is not None: + entry["pr"] = pr + entry["sha"] = landed(repo, args.sha, args.default) if args.status == "merged" else commit(repo, args.sha) + council = module("council", rasci.COUNCIL) + sid, _ = council.caller(args.session) + entry["roles"] = {council.live_sessions().get(sid) or sid: "RA"} + rasci.apply_status(entry, REGISTER_STATUS[args.status]) + with rasci.database() as tasks: + if task in tasks: + fail(f"{task} is already in the register; move it on with update") + tasks[task] = entry + print(f"added {task}") + return 0 + + +def located(task: str, entry: dict, given: str | None) -> tuple[str, dict]: + if given is None: + common = entry.get("common_dir") or "" + if not os.path.isdir(common): + fail(f"{task}: its git dir {common or '-'} is gone; pass --repo ") + return common, {} + repo = clone(given) + here = identity(repo) + theirs = recorded(entry) + if theirs is not None and not same(theirs, here): + fail(f"--repo {given} is another repository than {task}'s {repository(entry)}") + return repo, {"common_dir": here[0], "origin": here[1]} + + +def cmd_update(rasci: ModuleType, args: argparse.Namespace) -> int: + task = task_of(rasci, args.slug) + pr = number(args.pr) + if args.status == "merged" and args.sha is None: + fail("--status merged needs --sha, the commit as it sits on the default branch") + entry = rasci.load().get(task) + if entry is None or "stage" not in entry: + fail(f"{task}: no hotfix in the register; record it with add") + fields: dict = {} + if args.repo is not None or args.sha is not None: + where, fields = located(task, entry, args.repo) + if args.sha is not None: + fields["sha"] = landed(where, args.sha, args.default) if args.status == "merged" else commit(where, args.sha) + if pr is not None: + fields["pr"] = pr + with rasci.database() as tasks: + entry = tasks.get(task) + if entry is None or "stage" not in entry: + fail(f"{task}: no hotfix in the register; record it with add") + entry.update(fields) + entry["stage"] = args.status + rasci.apply_status(entry, REGISTER_STATUS[args.status]) + print(f"updated {task}") + return 0 + + +def cmd_pending(rasci: ModuleType, args: argparse.Namespace) -> int: + repo = worktree(args.repo) + rows, unknowns = of_repository(rasci, repo) + rows = for_targets(rows, args.target) + problems: list[str] = [] + on, off, shown = missing(repo, rows, problems) + verdicts = {t: "missing (merge main)" for t, _ in on} | {t: f"not on {shown}" for t, _ in off} + opened = 0 + for task, row in rows: + if row["stage"] != "merged": + verdict = "open, not merged" + opened += 1 + elif task in verdicts: + verdict = verdicts[task] + else: + continue + print("\t".join((verdict, task, row["stage"], row["branch"], row["sha"], pr_cell(row)))) + report([*unknowns, *problems]) + print(f"hotfix: {len(on)} missing, {opened} open") + if problems: + return 2 + return 1 if on else 0 + + +def cmd_match(rasci: ModuleType, args: argparse.Namespace) -> int: + here = identity(worktree(args.repo)) if args.repo is not None else None + rows = [] + for task, row in hotfixes(rasci): + try: + re.compile(row["signature"]) + except re.error as error: + print(f"hotfix: {task}: signature does not compile ({error}), skipped", file=sys.stderr) + continue + rows.append((task, row)) + hits = [] + while rows: + verdicts = searched([row["signature"] for _, row in rows], args.line) + hits += [entry for entry, hit in zip(rows[:len(verdicts)], verdicts, strict=True) if hit] + if len(verdicts) == len(rows): + break + task, row = rows[len(verdicts)] + print(f"hotfix: {task}: signature {row['signature']!r} still searched this line at the {SEARCH_SECONDS} s " + "deadline, skipped; its A must narrow it", file=sys.stderr) + rows = rows[len(verdicts) + 1:] + found = 0 + for task, row in hits: + cells = [task, row["stage"], row["branch"], row["sha"], pr_cell(row), repository(row)] + theirs = recorded(row) + if here is not None and theirs is None: + cells.insert(0, "unknown") + elif here is not None and not same(theirs, here): + cells.insert(0, "foreign") + else: + found += 1 + print("\t".join(cells)) + if not found: + print("hotfix: no signature matches" if here is None else "hotfix: no signature of this repository matches") + return 0 if found else 1 + + +def cmd_void(rasci: ModuleType, args: argparse.Namespace) -> int: + repo = worktree(args.repo) + rows, unknowns = of_repository(rasci, repo) + rows = for_targets(rows, args.target) + problems: list[str] = [] + on, off, shown = missing(repo, rows, problems) + for task, row in off: + print(f"hotfix: {task}: {row['sha']} is not on {shown} (a squash or rebase merge?), so it voids nothing; " + "its A must record the commit as it sits there", file=sys.stderr) + for task, row in on: + print("\t".join(("void", task, row["sha"], ",".join(row.get("targets") or []) or "all targets"))) + report([*unknowns, *problems]) + if on: + return 3 + return 2 if problems else 0 + + +@contextlib.contextmanager +def locked(repo: str, command: list[str]) -> Iterator[None]: + """Hold gate.py's sync lock and worktree lock of the worktree. + + Args: + repo: the worktree. + command: sync's command line, recorded in the sync lock. + """ + if not GATE.is_file(): + yield + return + gate = module("gate", GATE) + git_dir = Path(git_out(repo, "rev-parse", "--absolute-git-dir").strip()) + with ( + open(git_dir / gate.SYNC_LOCK, "a", encoding="utf-8") as sync, + open(git_dir / gate.RUN_LOCK, "a", encoding="utf-8") as run, + ): + fcntl.flock(sync, fcntl.LOCK_EX) + gate.claim(sync, command) + try: + fcntl.flock(run, fcntl.LOCK_EX | fcntl.LOCK_NB) + except BlockingIOError: + fail(f"a gate.py run or hold holds {repo} ({gate.holder(git_dir / gate.RUN_LOCK)}); sync once it ends") + yield + + +def watched(repo: str) -> None: + if WATCH.is_file(): + watch = module("watch", WATCH) + config = watch.load_config() + if config is not None and watch.busy(repo, config["busy_processes"]): + fail(f"a busy process (watch.py busy_processes) runs in {repo}, a gate or deploy; sync once it ends") + + +def chain(repo: str, tasks: dict[str, dict], task: str, default: str, ended: tuple[str, ...]) -> list[str]: + here = identity(repo) + places: dict[str, tuple[str, str] | None] = {} + found: list[str] = [] + base = tasks[task].get("base") + while base and base != default: + if base in found: + fail(f"base chain of {task} loops: {' <- '.join([*found, base])}") + found.append(branch_name(repo, base, "base")) + upstream = {} + for name, row in sorted(tasks.items()): + if row.get("branch") != base or not row.get("base") or row.get("status") in ended: + continue + workdir = row.get("workdir") or "" + if not workdir: + if not (resolves(repo, f"refs/heads/{base}") or resolves(repo, f"refs/remotes/origin/{base}")): + fail(f"{name} records no workdir and its branch {base} resolves neither as {base} nor as " + f"origin/{base} in {repo}; record it with `rasci.py set {name} --workdir `") + upstream[name] = row["base"] + continue + if workdir not in places: + places[workdir] = identity(workdir) + if places[workdir] is None: + print(f"hotfix: {name}: workdir {workdir} is no git repository, its base is not followed", + file=sys.stderr) + elif same(places[workdir], here): + upstream[name] = row["base"] + if len(set(upstream.values())) > 1: + named = ", ".join(f"{t}={b}" for t, b in sorted(upstream.items())) + fail(f"branch {base} has several bases in the register: {named}") + base = next(iter(upstream.values()), None) + return found + + +def base_ref(repo: str, base: str) -> tuple[str, str]: + local, remote = f"refs/heads/{base}", f"refs/remotes/origin/{base}" + if not resolves(repo, remote): + if not resolves(repo, local): + fail(f"{base} does not resolve in {repo}") + return base, local + if not resolves(repo, local) or ancestor(repo, local, remote): + return f"origin/{base}", remote + if ancestor(repo, remote, local): + return base, local + fail(f"{base} and origin/{base} have diverged in {repo}; merge one into the other first") + + +def default_ref(repo: str, given: str | None) -> tuple[str, tuple[str, str]]: + if git(repo, "config", "--get", "remote.origin.url").stdout.strip(): + default = default_branch(repo, given) + fetched(repo) + return default, (f"origin/{default}", f"refs/remotes/origin/{default}") + if given is None: + fail(f"{repo} has no origin remote; pass --default , its local default branch") + default = branch_name(repo, given, "--default") + return default, (default, f"refs/heads/{default}") + + +def cmd_sync(rasci: ModuleType, args: argparse.Namespace) -> int: + repo = worktree(args.repo) + tasks = rasci.load() + if args.task not in tasks: + fail(f"{args.task}: not in the register") + branch = tasks[args.task].get("branch") + if not branch: + fail(f"{args.task} records no branch; set it with `rasci.py set {args.task} --branch `") + head = git(repo, "symbolic-ref", "--quiet", "HEAD").stdout.strip() + if head != f"refs/heads/{branch}": + fail(f"{repo} has {head.removeprefix('refs/heads/') or 'a detached HEAD'} checked out, " + f"{args.task} works on {branch}") + default, first = default_ref(repo, args.default) + with locked(repo, ["hotfix.py", "sync", "--repo", repo, "--task", args.task]): + watched(repo) + dirty = [line[3:] for line in git_out(repo, "status", "--porcelain=v1", "--untracked-files=no").splitlines()] + if dirty: + fail(f"tracked files are dirty, commit them first: {', '.join(dirty)}") + if not resolves(repo, first[1]): + fail(f"{first[0]} does not resolve in {repo}") + bases = chain(repo, tasks, args.task, default, rasci.ENDED) + for shown, ref in [first, *(base_ref(repo, base) for base in bases)]: + if ancestor(repo, ref, "HEAD"): + print(f"contained\t{shown}") + continue + result = git(repo, "merge", "--no-verify", "--no-edit", ref) + if result.returncode == 0: + print(f"merged\t{shown}\t{git_out(repo, 'rev-parse', 'HEAD').strip()}") + continue + files = git(repo, "diff", "--name-only", "--diff-filter=U").stdout.splitlines() + if git(repo, "rev-parse", "--verify", "--quiet", "MERGE_HEAD").returncode == 0: + git_out(repo, "merge", "--abort") + if not files: + fail(f"git merge {shown} failed and was aborted: {(result.stderr or result.stdout).strip()}") + print(f"conflict\t{shown}\t{','.join(files)}") + print(f"hotfix: merging {shown} conflicts in {', '.join(files)}; merge aborted", file=sys.stderr) + return 4 + return 0 + + +def main() -> int: + parser = argparse.ArgumentParser(description=__doc__.splitlines()[0]) + sub = parser.add_subparsers(dest="cmd", required=True) + p = sub.add_parser("add") + p.add_argument("slug") + for flag in ("--repo", "--branch", "--sha", "--signature"): + p.add_argument(flag, required=True) + p.add_argument("--status", choices=STAGES, required=True) + p.add_argument("--target", action="append") + p.add_argument("--pr", type=int) + p.add_argument("--default") + p.add_argument("--session") + p.set_defaults(func=cmd_add) + p = sub.add_parser("update") + p.add_argument("slug") + p.add_argument("--status", choices=STAGES, required=True) + p.add_argument("--sha") + p.add_argument("--pr", type=int) + p.add_argument("--repo") + p.add_argument("--default") + p.set_defaults(func=cmd_update) + p = sub.add_parser("pending") + p.add_argument("--repo", required=True) + p.add_argument("--target", action="append") + p.set_defaults(func=cmd_pending) + p = sub.add_parser("match") + p.add_argument("--line", required=True) + p.add_argument("--repo") + p.set_defaults(func=cmd_match) + p = sub.add_parser("void") + p.add_argument("--repo", required=True) + p.add_argument("--target", action="append", required=True) + p.set_defaults(func=cmd_void) + p = sub.add_parser("sync") + p.add_argument("--repo", required=True) + p.add_argument("--task", required=True) + p.add_argument("--default") + p.set_defaults(func=cmd_sync) + args = parser.parse_args() + return args.func(module("rasci", RASCI), args) + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/skills/kvb-org-coordinate/SKILL.md b/skills/kvb-org-coordinate/SKILL.md index c2660c0..4ddb9e7 100644 --- a/skills/kvb-org-coordinate/SKILL.md +++ b/skills/kvb-org-coordinate/SKILL.md @@ -8,5 +8,8 @@ description: > Run the `kvb-org-organize` skill's **Peer protocol**, its `lineage.md` included, for the failure or fix at hand. They hold the whole procedure, -including the entries in the `kvb-org-rasci` register: ask by `SendMessage`, -report by `news.py post`. This skill only keeps its command working. +including the entries in the `kvb-org-rasci` register: look first, `hotfix.py +match` of the `kvb-git-hotfix` skill included; ask by `SendMessage`; report +by `news.py post`. A hotfix is recorded with `hotfix.py add` and `update`, +never reported by message or news post. This skill only keeps its command +working. diff --git a/skills/kvb-org-organize/SKILL.md b/skills/kvb-org-organize/SKILL.md index 5615d72..bc970b9 100644 --- a/skills/kvb-org-organize/SKILL.md +++ b/skills/kvb-org-organize/SKILL.md @@ -7,7 +7,7 @@ description: > --- The first claim owns a piece of work; a later agent hands its findings to the -owner and never builds a second version. Offices are recallable mandates +owner, never building a second version. Offices are recallable mandates that coordinate and allocate, never direct others' content; the operator outranks them and appoints or recalls at will. @@ -18,15 +18,15 @@ outranks them and appoints or recalls at will. 1. Name the office: `work/` (a piece of work), `cluster/` (a strand across lines), `/` (a line) or `chair`, the slug - concrete (issue number, failing line, log line). The first agent elected + concrete (issue number, failing log line). The first agent elected while `chair` is vacant also claims it. Read `offices.md` next to this file before claiming a cluster, a line or the chair. 2. `council.py list`; the same work claimed under another slug counts. 3. `council.py claim --work ""`. - Exit 0: yours; 3: a live holder (printed); 4: the holder's session is dead. + Exit 0: yours; 3: a live holder (printed); 4: a dead holder. 4. Elected: complete the task the claim entered in the `kvb-org-rasci` - register (objective, description, priority, roles); the listed claim is - the announcement. + register (objective, description, priority, roles); the listed claim + announces it. 5. Not elected: post your findings to the holder via the news and stop. A hook (`council.py gate`) blocks worktree edits before a claim. Between @@ -42,9 +42,11 @@ ends or goes stale, or after a restart. - `make test`, `make test-lint`, quality gates, deploys, iteration loops, swarm, image builds, cloud, shared-data cleanup, anything with Docker or over a few minutes: ask the chair (what, where, how long, disk), wait for - the grant, post the release to the news. + the grant, post the release to the news. In a worktree they run through + `gate.py run` or `gate.py hold` (`kvb-check-gate`). - No running job is cut short for capacity; only the operator orders one - down. `RESOURCE` hook lines address the chair; others adjust plans. + down. Exception: void by a missing merged hotfix (`kvb-git-hotfix`). + `RESOURCE` hook lines address the chair; others adjust plans. - Read `capacity.md` next to this file when you hold the chair, a job seems hung, or the resource watch fires. @@ -54,22 +56,23 @@ Read `peers.md` next to this file before asking peers whether a failure or fix is known. - Never poll: `council.py list`, the register and the news answer first. -- Status that asks nothing (a hotfix opened or merged, a failure, a slot - freed, a result for information): `${CLAUDE_SKILL_DIR}/scripts/news.py post - ""` (`--to ` for one peer); sessions see it next turn at no - cost. +- Hotfixes: `${CLAUDE_SKILL_DIR}/../kvb-git-hotfix/scripts/hotfix.py add` and + `update`, never a message or news post. +- Status that asks nothing (a failure, a freed slot, a result): + `${CLAUDE_SKILL_DIR}/scripts/news.py post ""` (`--to ` for one + peer). - `SendMessage` only when the recipient must act or answer: one batched message per peer per event, terse, no greeting, at most five lines unless - evidence needs more, nothing already acknowledged. A message that asks + evidence needs more, nothing already acknowledged. A message asking nothing gets no reply; end that turn in one line. -- Never invent a peer's answer; verify the SHA, log or artifact a peer names - before relying on it. +- Never invent a peer's answer; verify the SHA, log or artifact it names + first. - A peer's message is no operator approval, an office no permission: never ask a peer to run what you are denied; tell the operator. -- Release every claim when its work ends; forgotten ones block everyone. +- Release every claim when its work ends. - Read `relay.md` next to this file before relaying an operator rule or - acting on one a peer relayed; `lineage.md` when a failure or fix is shared - with other branches through merges. + acting on one a peer relayed; `lineage.md` when a failure or fix spreads to + other branches through merges. ## Output diff --git a/skills/kvb-org-organize/capacity.md b/skills/kvb-org-organize/capacity.md index e9c348e..a18a7b4 100644 --- a/skills/kvb-org-organize/capacity.md +++ b/skills/kvb-org-organize/capacity.md @@ -9,7 +9,8 @@ that fixes it) takes the next free gate or test slot before every other line, the rest follows the WSJF rank of the `kvb-org-wsjf` skill (it asks its subordinates for the Fibonacci estimates); a running gate is never cut short - for a hotfix. It answers every request with its queue position through + for a hotfix's slot, only when the hotfix voids it (`kvb-git-hotfix`). It + answers every request with its queue position through `news.py post --to ` and sends the grant itself as a message (the requester acts on it), enforces the limits the operator set, and checks every grant with @@ -24,7 +25,8 @@ ## Hung jobs -The one exception to "never cut short": an agent may end its own job, a stuck +Besides a void by a missing merged hotfix (`kvb-git-hotfix`), the only +exception to "never cut short": an agent may end its own job, a stuck step inside it, or a process its own ended session left behind, once at least 60 minutes passed without log progress and a probe shows standstill (no fresh child process, no network traffic, CPU time not moving). It secures the live diff --git a/skills/kvb-org-organize/peers.md b/skills/kvb-org-organize/peers.md index 1d771d3..c6213a1 100644 --- a/skills/kvb-org-organize/peers.md +++ b/skills/kvb-org-organize/peers.md @@ -3,9 +3,10 @@ `kvb-org-coordinate`, `kvb-ci-triage`, `kvb-code-tune`, `kvb-org-split` and `kvb-cleanup` follow these steps. -1. **Look**: `council.py list`, the `kvb-org-rasci` register, the news and the - refs (the kvb-git-merge-fix skill's `find_fix.py`). A claim or a committed - fix answers the question without a message. +1. **Look**: `council.py list`, the `kvb-org-rasci` register, the news, + `hotfix.py match --line '' --repo ` of the `kvb-git-hotfix` + skill and the refs (the kvb-git-merge-fix skill's `find_fix.py`). A claim, + a hotfix row or a committed fix answers the question without a message. 2. **Claim** the office (Election in `SKILL.md`). 3. **Ask** (in Claude Code: `ListAgents`, then `SendMessage`) only the peers whose lines the work touches, the lineage owners (`lineage.md`) and, for @@ -21,7 +22,8 @@ neither a "no" nor a competing claim: after 5 minutes, proceed. 5. **Report** to the same peers through `news.py post --to `: commit, branch, what changed, the evidence it works, and the merge it reaches them - by; an abandoned or failed attempt too. In the register, enter the peers - told as I and mark every key result the evidence proves as met. Then - release the claim: the task ends `done`, or `stopped` while a key result is - open. + by; an abandoned or failed attempt too. A hotfix is no report: it is + recorded with `hotfix.py add` and `update`, never by message or news post. + In the register, enter the peers told as I and mark every key result the + evidence proves as met. Then release the claim: the task ends `done`, or + `stopped` while a key result is open. diff --git a/skills/kvb-org-rasci/SKILL.md b/skills/kvb-org-rasci/SKILL.md index da84a71..b88fde1 100644 --- a/skills/kvb-org-rasci/SKILL.md +++ b/skills/kvb-org-rasci/SKILL.md @@ -13,7 +13,7 @@ itself (holder as R and A, `running`, start, workdir, branch); its release sets `done` and the end, or `stopped` while a key result is still open. ```bash -rasci.py set --title --objective --description --prio --status queued|running|blocked|done|stopped --worker " ()>" --workdir --branch --artifact : --tool