kevinveenbirkenbach/skills
1
0 Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity

feat(kvb-git-hotfix): share hotfixes in the register and sync before every action

Browse Source 
The Maintainer messaged every agent about every hotfix, waking each for a full turn. Hotfixes are now register rows hotfix/<slug> (repository identity by git common dir and normalized origin URL, branch, sha, PR, failing-line signature, targets, stage), written with hotfix.py add/update and queried only when an agent acts, never by cron: pending lists merged hotfixes missing from HEAD (filtered by targets) and open ones; match finds hotfixes whose signature fits a failing line, evaluated in one worker under a shared deadline; void tells gate.py run, before every target, when a merged hotfix missing from HEAD makes the rest of the run worthless (exit 3), the operator's one exception to never cutting a job short; sync fetches first, then under the worktree locks merges origin/<default> and every base up the funnel chain recorded with rasci.py set --base, refusing a detached or foreign branch, a diverged base or a busy worktree, aborting cleanly on a conflict.

gate.py gains hold, which runs any heavy command, deploys included, under the worktree lock, forwards signals, returns the command's code and exits 75 when busy; run holds the same lock and takes the sync lock shared. kvb-check-gate, kvb-ci-triage, kvb-org-organize (Peer protocol, capacity), kvb-org-coordinate and kvb-org-split follow; rasci gaps, wsjf and artifacts skip hotfix and operator rows. Three independent skeptic rounds found 34 defects (three of them high), all fixed with tests; negative controls fail against the earlier code.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Kevin Veen-BirkenbachandClaude Opus 5.5 committed 2026-11-04 17:07:23 +01:00
1 parent 713267dd15
commit b4c4cefadd
17 files changed
+2497 -115
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

README.md
+6
View File
@@ -57,6 +57,7 @@ flowchart LR
git --> git_update_branches[kvb-git-update-branches]
git --> git_branch_gc[kvb-git-branch-gc]
git --> git_changelog[kvb-git-changelog]
git --> git_hotfix[kvb-git-hotfix]
check --> check_legal[kvb-check-legal]
check --> check_security[kvb-check-security]
check --> check_principles[kvb-check-principles]
@@ -149,6 +150,7 @@ flowchart LR
update_branches[kvb-git-update-branches]
branch_gc[kvb-git-branch-gc]
changelog[kvb-git-changelog]
hotfix[kvb-git-hotfix]
end
subgraph cat_check [check]
advocate[kvb-check-legal]
@@ -232,6 +234,10 @@ flowchart LR
confidence --> gate
pr_label --> gate
gate --> democracy
gate --> hotfix
triage --> hotfix
democracy --> hotfix
hotfix --> rasci
confidence -. objective .-> autotune
coordinate --> democracy
skills/kvb-check-gate/SKILL.md
+35 -21
View File
@@ -11,26 +11,30 @@ target of the project, each Makefile target named `test` or starting with
`test-`, ran green on the exact head with a clean tracked tree. Until then
the numbers are provisional and every target not green is a residual.
`${CLAUDE_SKILL_DIR}/scripts/gate.py` is the deterministic part. It reads the
targets from the Makefile and every file it includes without running make,
and keeps the results per commit in `kvb-gate.json` in the git common dir:
linked worktrees share them, a new commit starts from zero.
`${CLAUDE_SKILL_DIR}/scripts/gate.py` reads the targets from the Makefile and
every file it includes without running make, and keeps the results per
commit in `kvb-gate.json` in the git common dir: linked worktrees share them,
a new commit starts from zero.
## Steps
1. **List**: `gate.py list --repo <root>`. A warning about an unresolved
1. **Sync**: `${CLAUDE_SKILL_DIR}/../kvb-git-hotfix/scripts/hotfix.py sync
--repo <root> --task <your register task>`, then `hotfix.py pending
--repo <root>`, by the `kvb-git-hotfix` skill. A merge moves HEAD, so the
gate starts over from there.
2. **List**: `gate.py list --repo <root>`. A warning about an unresolved
include or target expression means the list may be short: read that line
and name the gap. A project without any target keeps the gate open; ask
the operator.
2. **Status**: `gate.py status --repo <root>`. `gate: green` ends here.
3. **Admit**: every gate target is heavy (`make test` and its siblings run
whole suites, often containers). Ask the resource chair of the
3. **Status**: `gate.py status --repo <root>`. `gate: green` ends here; a
`missing hotfix` reason means back to step 1.
4. **Admit**: every gate target is heavy. Ask the resource chair of the
`kvb-org-organize` skill per target (what, where, how long, disk), wait
for the grant, post the release to its news. Only a single test module
self-admits (`${CLAUDE_SKILL_DIR}/../kvb-org-organize/scripts/watch.py
admit --kind test-nostack`), and it never turns a gate target green. A
hotfix takes the next free gate slot.
4. **Run** the missing and red targets, tracked tree clean, in the
5. **Run** the missing and red targets, tracked tree clean, in the
background; give the operator `tail -f` on the log:
```bash
@@ -38,22 +42,32 @@ linked worktrees share them, a new commit starts from zero.
```
Pass one `--fail-regex` per failure line the project prints while exiting
0; infinito-nexus needs `FAILED TARGETS: \S`. A target is red on a
non-zero exit, a `make: ***` line, a matching regex, or tracked files it
modified; the last one stops the run.
5. **Judge** every red target by the line `status` prints and the log around
it: caused by the change, already red on the default branch, or the
environment. Fix the cause and run again; a red target never counts as
green because it looks unrelated.
6. **Status** once more; its last line is the verdict.
0; infinito-nexus needs `FAILED TARGETS: \S`. A non-zero exit, a
`make: ***` line, a matching regex or modified tracked files make a
target red; the last stops the run. Before each target `run` asks
`hotfix.py void` for it and every later one. Exit 3 (`void: merge main
for hotfix <slug>, then rerun`): post the release to the news, back to
step 1, and the chair's grant again at step 4. Exit 75: the worktree is
busy, the message names the holder; wait, then rerun.
6. **Judge** every red target by its `status` line and the log around it:
caused by the change, already red on the default branch, or the
environment. Fix the cause and rerun; looking unrelated never makes it
green.
7. **Status** once more; its last line is the verdict.
## Rules
- Only `gate.py run` on this head marks a target green. A peer's report, a
CI run, an older commit or a run on a dirty tree never does.
- Only `gate.py run` on this head marks a target green. A peer's report, CI,
an older commit or a dirty tree never does.
- Every heavy command in a worktree (deploys, `make test`, other gates) runs
through `gate.py run` or `gate.py hold --repo <root> -- <command>` in the
background (`run_in_background`), never under a tool timeout or
`TaskStop`: it is never cut short. Their lock keeps `hotfix.py sync` out
only while a host process under it lives; a `docker compose exec` deploy
runs on in its container unguarded.
- 90/5 is allowed only on `gate: green` for the exact head: in the
`kvb-check-confidence` block, the `90/5` label of the `kvb-git-pr-label`
skill and the bar of the `kvb-org-split` skill.
`kvb-check-confidence` block, the `90/5` label of `kvb-git-pr-label` and
the bar of `kvb-org-split`.
- Never edit `kvb-gate.json` by hand.
## Output
skills/kvb-check-gate/scripts/gate.py
+257 -50
View File
@@ -6,41 +6,82 @@ Args:
parsed statically from the Makefile make reads in <dir> and every file
it includes; make is never run.
status --repo <dir> [--json]: per listed target green, red or missing for
HEAD, from `kvb-gate.json` in the git common dir.
HEAD, from `kvb-gate.json` in the git common dir; `hotfix.py pending
--target <t> ...` of the kvb-git-hotfix skill, given the listed
targets, keeps the gate open while a merged hotfix that turns one of
them green is missing from HEAD.
run --repo <dir> --target <t> [--target ...] --log-dir <dir>
[--fail-regex <re> ...]: run each target as `make -C <dir> <t>` with
stdout and stderr in <log-dir>/gate-<t>-<sha8>.log and record the
result for HEAD. A target is red on a non-zero exit, a `make: ***`
line, a matching --fail-regex, or tracked files it modified.
[--fail-regex <re> ...]: under the worktree lock, run each target as
`make -C <dir> <t>` with stdout and stderr in
<log-dir>/gate-<t>-<sha8>.log and record the result for HEAD. A
target is red on a non-zero exit, a `make: ***` line, a matching
--fail-regex, or tracked files it modified. Before each target,
`hotfix.py void` decides over it and every target after it whether a
merged hotfix missing from HEAD voids the run.
hold --repo <dir> -- <command...>: under the worktree lock, run the
command in <dir> in a session of its own, without a controlling
terminal; SIGINT, SIGTERM and SIGHUP reaching hold go to the
command's process group while hold keeps waiting for it. For deploys
and every other heavy command that is no `make test*` target.
The worktree lock is `kvb-gate-run.lock` in the worktree's git dir,
taken without waiting and passed to make or the command, so it stays
held while they or a process they left running on this host live; a
process inside a container (`docker compose exec`) runs on there after
they die, and the lock no longer covers it. run and hold write their PID
and command into it, and a busy message names them. `hotfix.py sync`
takes `kvb-gate-sync.lock` exclusively, recording its PID and command
there, then the worktree lock, and releases them in reverse order; run
and hold take the worktree lock while holding `kvb-gate-sync.lock`
shared, so they wait for a running sync, naming it, for at most
SYNC_WAIT_SECONDS, and a busy worktree lock then means another run or
hold.
Without hotfix.py next to this skill, or when it exits non-zero without
a verdict line, gate.py warns on stderr and goes on without the hotfix
check.
Output:
list: one `<target>\t<file>:<line>` per target.
status: one `<status>\t<target>\t<log>\t<finished>\t<reason>` per target,
then `gate: green`, or `gate: open (...)` naming what is missing.
run: one `<status>\t<target>\t<log>\t<reason>` per target.
then `gate: green`, or `gate: open (...)` naming what is missing,
`missing hotfix: <slug>, ...` included.
run: one `<status>\t<target>\t<log>\t<reason>` per target; on a void
`void: merge main for hotfix <slug>, then rerun`.
hold: the command's own output.
--json prints the same data as JSON.
Exit codes:
list: 0 done, 2 usage error.
status: 0 gate green, 1 gate open, 2 usage error.
run: 0 every target green, 1 a target red, 2 usage error, dirty tracked
files or HEAD moved during a run.
files or HEAD moved during a run, 3 void: the remaining targets are
neither run nor recorded, 75 (EX_TEMPFAIL) busy: another run or hold
holds the worktree, or a sync still holds it after SYNC_WAIT_SECONDS.
hold: the command's exit code unchanged, 128+N when signal N ended it;
before the command started 2 on a usage error, 75 busy as for run,
128+N when signal N reached hold.
"""
from __future__ import annotations
import argparse
import contextlib
import fcntl
import glob
import json
import os
import re
import shlex
import signal
import subprocess
import sys
import tempfile
import time
from collections.abc import Iterator
from datetime import UTC, datetime
from pathlib import Path
from typing import NoReturn
from typing import NoReturn, TextIO
MAKEFILES = ("GNUmakefile", "makefile", "Makefile")
INCLUDE = re.compile(r"^\s*(?P<kind>-include|sinclude|include)\s+(?P<paths>.+)$")
@@ -55,6 +96,13 @@ MAKE_ERROR = re.compile(r"\bmake(?:\[\d+\])?: \*\*\*")
UNSAFE = re.compile(r"[^\w.-]")
LINE_CHARS = 300
STATE = "kvb-gate.json"
RUN_LOCK = "kvb-gate-run.lock"
SYNC_LOCK = "kvb-gate-sync.lock"
SYNC_WAIT_SECONDS = 300
POLL_SECONDS = 0.1
BUSY = os.EX_TEMPFAIL
FORWARDED = (signal.SIGINT, signal.SIGTERM, signal.SIGHUP)
HOTFIX = Path(__file__).resolve().parents[2] / "kvb-git-hotfix" / "scripts" / "hotfix.py"
def fail(message: str) -> NoReturn:
@@ -62,6 +110,11 @@ def fail(message: str) -> NoReturn:
sys.exit(2)
def busy(message: str) -> NoReturn:
print(f"gate: {message}", file=sys.stderr)
sys.exit(BUSY)
def logical_lines(path: Path) -> list[tuple[int, str]]:
lines: list[tuple[int, str]] = []
start, buffer = 0, ""
@@ -212,6 +265,114 @@ def first_failure(log: Path, checks: list[re.Pattern]) -> str | None:
return None
def hotfix_checks() -> bool:
if HOTFIX.is_file():
return True
print(f"gate: {HOTFIX} is missing; merged hotfixes are not checked", file=sys.stderr)
return False
def hotfix(repo: Path, command: str, args: list[str], verdict: str) -> list[str]:
result = subprocess.run(
[sys.executable, str(HOTFIX), command, *args, "--repo", str(repo)],
stdin=subprocess.DEVNULL, capture_output=True, text=True, check=False,
)
sys.stderr.write(result.stderr)
slugs = [
line.split("\t")[1].removeprefix("hotfix/")
for line in result.stdout.splitlines() if line.startswith(f"{verdict}\t")
]
if result.returncode and not slugs:
print(f"gate: hotfix.py {command} exited {result.returncode}; going on without its check", file=sys.stderr)
return slugs
def target_flags(targets: list[str]) -> list[str]:
return [part for target in targets for part in ("--target", target)]
def voiding_hotfixes(repo: Path, targets: list[str]) -> list[str]:
return hotfix(repo, "void", target_flags(targets), "void")
def missing_hotfixes(repo: Path, targets: list[str]) -> list[str]:
return hotfix(repo, "pending", target_flags(targets), "missing (merge main)")
def claim(lock: TextIO, command: list[str]) -> None:
"""Record the holder in a lock file it holds exclusively.
Args:
lock: the open, exclusively held lock file.
command: the holder's command line.
"""
lock.truncate(0)
lock.write(f"{os.getpid()}\t{shlex.join(command)}\n")
lock.flush()
def holder(path: Path) -> str:
"""Name the holder that `claim` recorded in a lock file.
Args:
path: the lock file.
"""
try:
pid, _, command = path.read_text(encoding="utf-8").strip().partition("\t")
except OSError:
pid, command = "", ""
return f"pid {pid}: {command}" if pid and command else "holder not recorded"
def shared(sync: TextIO, repo: Path, caught: list[int]) -> None:
"""Take SYNC_LOCK shared, waiting at most SYNC_WAIT_SECONDS for a running sync.
Args:
sync: the open SYNC_LOCK file.
repo: the worktree, named in messages.
caught: signals the caller caught; the first ends the wait with 128+N.
"""
deadline = time.monotonic() + SYNC_WAIT_SECONDS
waiting = False
while True:
try:
fcntl.flock(sync, fcntl.LOCK_SH | fcntl.LOCK_NB)
return
except BlockingIOError:
pass
if caught:
sys.exit(128 + caught[0])
if time.monotonic() >= deadline:
busy(f"a hotfix.py sync still holds {repo} after {SYNC_WAIT_SECONDS} s ({holder(Path(sync.name))}); "
"retry once it ends")
if not waiting:
print(f"gate: a hotfix.py sync merges in {repo} ({holder(Path(sync.name))}); waiting for it at most "
f"{SYNC_WAIT_SECONDS} s", file=sys.stderr, flush=True)
waiting = True
time.sleep(POLL_SECONDS)
@contextlib.contextmanager
def worktree_lock(repo: Path, command: list[str], caught: list[int]) -> Iterator[int]:
"""Hold the worktree lock; yields its file descriptor for pass_fds.
Args:
repo: the worktree whose git dir holds RUN_LOCK and SYNC_LOCK.
command: the holder's command line, recorded in RUN_LOCK.
caught: signals the caller caught; the first ends a wait for a sync with 128+N.
"""
git_dir = Path(git(repo, "rev-parse", "--absolute-git-dir").strip())
with open(git_dir / RUN_LOCK, "a", encoding="utf-8") as lock:
with open(git_dir / SYNC_LOCK, "a", encoding="utf-8") as sync:
shared(sync, repo, caught)
try:
fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB)
except BlockingIOError:
busy(f"another gate.py run or hold holds {repo} ({holder(git_dir / RUN_LOCK)})")
claim(lock, command)
yield lock.fileno()
def repository(value: str) -> Path:
repo = Path(value).resolve()
if not repo.is_dir():
@@ -254,6 +415,9 @@ def cmd_status(args: argparse.Namespace) -> int:
reasons.append(f"red: {', '.join(red)}")
if dirty:
reasons.append(f"dirty: {len(dirty)} tracked file(s)")
hotfixes = missing_hotfixes(repo, list(found)) if hotfix_checks() else []
if hotfixes:
reasons.append(f"missing hotfix: {', '.join(hotfixes)}")
gate = "open" if reasons else "green"
if args.json:
print(json.dumps(
@@ -283,51 +447,90 @@ def cmd_run(args: argparse.Namespace) -> int:
unknown = [t for t in args.target if t not in found]
if unknown:
fail(f"not a make test* target of {repo}: {', '.join(unknown)}")
sha = head(repo)
dirty, untracked = tree(repo)
if dirty:
fail(f"tracked files are dirty, commit or drop them first: {', '.join(dirty)}")
if untracked:
print(f"untracked\t{len(untracked)} file(s), not part of the gate")
log_dir = Path(args.log_dir).resolve()
log_dir.mkdir(parents=True, exist_ok=True)
state = state_path(repo)
red = False
for target in dict.fromkeys(args.target):
log = log_dir / f"gate-{UNSAFE.sub('_', target)}-{sha[:8]}.log"
with open(log, "w", encoding="utf-8") as out:
code = subprocess.run(
["make", "-C", str(repo), target],
stdin=subprocess.DEVNULL, stdout=out, stderr=subprocess.STDOUT, check=False,
).returncode
if head(repo) != sha:
fail(f"HEAD moved during {target}; nothing recorded for it")
changed, _ = tree(repo)
reasons = []
if code:
reasons.append(f"exit {code}")
line = first_failure(log, checks)
if line is not None:
reasons.append(line)
if changed:
reasons.append(f"modified tracked files: {', '.join(changed)}")
status = "red" if reasons else "green"
red = red or status == "red"
entry = {"status": status, "log": str(log),
"finished": datetime.now(UTC).isoformat(timespec="seconds"),
"fail_regex": [c.pattern for c in checks]}
if reasons:
entry["reason"] = "; ".join(reasons)
record(state, sha, target, entry)
print("\t".join([status, target, str(log), cell(entry.get("reason"))]))
if changed:
fail(f"{target} left tracked files dirty; remaining targets not run")
with worktree_lock(repo, ["gate.py", "run", "--repo", str(repo), *target_flags(args.target)], []) as lock:
sha = head(repo)
dirty, untracked = tree(repo)
if dirty:
fail(f"tracked files are dirty, commit or drop them first: {', '.join(dirty)}")
if untracked:
print(f"untracked\t{len(untracked)} file(s), not part of the gate")
log_dir = Path(args.log_dir).resolve()
log_dir.mkdir(parents=True, exist_ok=True)
state = state_path(repo)
red = False
order = list(dict.fromkeys(args.target))
checking = hotfix_checks()
for index, target in enumerate(order):
slugs = voiding_hotfixes(repo, order[index:]) if checking else []
if slugs:
print(f"void: merge main for hotfix {', '.join(slugs)}, then rerun")
return 3
log = log_dir / f"gate-{UNSAFE.sub('_', target)}-{sha[:8]}.log"
with open(log, "w", encoding="utf-8") as out:
code = subprocess.run(
["make", "-C", str(repo), target],
stdin=subprocess.DEVNULL, stdout=out, stderr=subprocess.STDOUT, pass_fds=(lock,), check=False,
).returncode
if head(repo) != sha:
fail(f"HEAD moved during {target}; nothing recorded for it")
changed, _ = tree(repo)
reasons = []
if code:
reasons.append(f"exit {code}")
line = first_failure(log, checks)
if line is not None:
reasons.append(line)
if changed:
reasons.append(f"modified tracked files: {', '.join(changed)}")
status = "red" if reasons else "green"
red = red or status == "red"
entry = {"status": status, "log": str(log),
"finished": datetime.now(UTC).isoformat(timespec="seconds"),
"fail_regex": [c.pattern for c in checks]}
if reasons:
entry["reason"] = "; ".join(reasons)
record(state, sha, target, entry)
print("\t".join([status, target, str(log), cell(entry.get("reason"))]))
if changed:
fail(f"{target} left tracked files dirty; remaining targets not run")
return 1 if red else 0
def send(child: subprocess.Popen, signum: int) -> None:
if child.returncode is None:
with contextlib.suppress(ProcessLookupError, PermissionError):
os.killpg(child.pid, signum)
def cmd_hold(args: argparse.Namespace) -> int:
repo = repository(args.repo)
caught: list[int] = []
child: subprocess.Popen | None = None
def forward(signum: int, _frame: object) -> None:
if child is None:
caught.append(signum)
else:
send(child, signum)
for signum in FORWARDED:
signal.signal(signum, forward)
with worktree_lock(repo, ["gate.py", "hold", "--repo", str(repo), "--", *args.command], caught) as lock:
if caught:
return 128 + caught[0]
try:
child = subprocess.Popen(args.command, cwd=repo, pass_fds=(lock,), start_new_session=True)
except OSError as error:
fail(f"cannot start {args.command[0]}: {error.strerror}")
for signum in caught:
send(child, signum)
code = child.wait()
return code if code >= 0 else 128 - code
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
sub = parser.add_subparsers(dest="command", required=True)
sub = parser.add_subparsers(dest="subcommand", required=True)
for name in ("list", "status"):
command = sub.add_parser(name)
command.add_argument("--repo", required=True)
@@ -337,8 +540,12 @@ def main() -> int:
run.add_argument("--target", action="append", required=True)
run.add_argument("--log-dir", required=True)
run.add_argument("--fail-regex", action="append")
hold = sub.add_parser("hold")
hold.add_argument("--repo", required=True)
hold.add_argument("command", nargs="+")
args = parser.parse_args()
return {"list": cmd_list, "status": cmd_status, "run": cmd_run}[args.command](args)
commands = {"list": cmd_list, "status": cmd_status, "run": cmd_run, "hold": cmd_hold}
return commands[args.subcommand](args)
if __name__ == "__main__":
skills/kvb-ci-triage/SKILL.md
+3 -2
View File
@@ -23,8 +23,9 @@ steps 0 to 2 in one call; the judgement stays yours.
differ beyond interpolated identifiers (host, app, port, path) are
separate clusters.
3. **Coordinate** each cluster via the `kvb-org-organize` Peer protocol,
lineage included: claim it; read `council.py list`, the register, the news
and `find_fix.py`; then ask every live peer for known fixes, one batched
lineage included: `${CLAUDE_SKILL_DIR}/../kvb-git-hotfix/scripts/hotfix.py
match --line '<line>'` first; claim it; read `council.py list`, the
register, the news and `find_fix.py`; ask every live peer, one batched
`SendMessage` per peer for all clusters nothing there answers. Investigate
while waiting; take their fix or keep yours. Register (`kvb-org-rasci`):
objective the root cause fixed, one key result per failing job passing.
skills/kvb-git-hotfix/SKILL.md
+78
View File
@@ -0,0 +1,78 @@
---
name: kvb-git-hotfix
description: >
Share hotfixes through a register table instead of messages; merge main and
upstream branches before every deploy, merge or gate; match failures first.
---
`${CLAUDE_SKILL_DIR}/scripts/hotfix.py` keeps each hotfix as a task
`hotfix/<slug>` of the `kvb-org-rasci` register, the author as R and A;
never a message or news post.
```bash
hotfix.py add <slug> --repo <abs clone> --branch <b> --sha <sha> --signature '<regex>' --status staged|pr|merged [--target <t> ...] [--pr <n>]
hotfix.py update <slug> --status staged|pr|merged [--sha <sha>] [--pr <n>] [--repo <abs clone>]
hotfix.py sync --repo <worktree> --task <register task> [--default <branch>]
hotfix.py pending --repo <worktree>
hotfix.py match --line '<failing log line>' --repo <worktree>
hotfix.py void --repo <worktree> --target <t> [--target ...]
```
## Author: record, never announce
1. Fix committed on its branch: `add`. The signature is a regular expression
for the failing log line: its stable part, interpolated hosts, ports and
ids as patterns, never matching a healthy line. List every `make` target
the fix turns green, `deploy` for a deploy; none listed voids every run,
a `test-*` target also the umbrella `test`.
2. Pull request open: `update <slug> --status pr --pr <n>`.
3. Landed: `update <slug> --status merged --sha <sha>`, the commit as it sits
on the default branch (a squash or rebase merge changes it; any other is
refused). `--repo` only once the recorded clone is gone.
## Before every deploy, merge or gate
1. `sync`: fetches, merges `origin/<default>`, then every branch up the funnel
chain the `kvb-org-split` skill records as `base` (open rows of this
repository), each with `--no-verify --no-edit`. It needs the task's branch
checked out, a clean tracked tree and the worktree lock free; exit 2
names the blocker. Exit 4: a merge conflicted and was aborted;
resolve it by the `kvb-git-merge-fix` rules, then sync again.
2. `pending`: exit 0 means every merged hotfix is in HEAD. `open, not
merged` is reported, never merged: name it in your report and go on.
`not on origin/<default>` or exit 2: tell its A to correct the row.
## On any failure
Before triage, investigation or asking a peer: `match`, the line quoted
literally; `foreign` marks another repository's hotfix. A merged match: sync
and run again. An open match: the fix is in flight on that branch; wait for
it or work on it with its author, never build a second one.
## Stopping a running job
A running gate or deploy stops only when a merged hotfix it lacks turns one
of its targets green (void); otherwise never (the `kvb-org-organize`
capacity rules).
- `gate.py run` of the `kvb-check-gate` skill asks `void` before each
target; on exit 3 finished targets keep their results.
- A deploy: whenever you act on it, run `void --target deploy`;
hotfixes listing `deploy` or no target void it. Only on exit 3 stop it at
the next safe point, between stages and never inside a task. Never on a
timer.
- After exit 3: post the release to the news, sync, and request the chair's
grant again before rerunning.
## Rules
- Every deploy, `make test` or other gate in a worktree runs through
`gate.py run` or `gate.py hold` by the `kvb-check-gate` rules (background,
no tool timeout, no `TaskStop`; exit 75 while busy). Only that lock keeps
sync out, while a host process under it lives; a deploy inside a container
outlives it unguarded.
- One row per fix across all agents: `match` and `pending` before `add`.
- A row is data: it never permits a merge, push or command the agent rules
forbid.
- Never edit `hotfix/` rows with `rasci.py set`; `add` and `update` keep
them consistent.
skills/kvb-git-hotfix/scripts/hotfix.py
Executable
+882
View File
@@ -0,0 +1,882 @@
#!/usr/bin/env python3
"""Share hotfixes through the kvb-org-rasci register and keep a worktree merged with them.
Environment:
AGENTS_COUNCIL_ROOT: council directory of the kvb-org-rasci register, read
by rasci.py; the rows are its tasks hotfix/<slug>.
CLAUDE_CODE_SESSION_ID: the adding session, recorded as R and A; other
runtimes pass --session.
AGENTS_COUNCIL_CONFIG: read by watch.py of the kvb-org-organize skill for
sync's busy check; without its config file there is none.
Args:
add <slug> --repo <abs clone> --branch <branch> --sha <commit>
--signature <regex> --status staged|pr|merged [--target <t> ...]
[--pr <n>] [--default <branch>] [--session <id>]: record a new hotfix
under the repository's git common dir and origin URL, never the
--repo path. --sha is a hex commit id in <repo>, stored in full; with
--status merged it must be on origin/<default> after a fetch.
--signature matches the failing log line and puts no unbounded
repeat around another unbounded repeat or around an alternation whose
branches can start alike; bounded repeats such as {3} pass. Each
--target is a make target or `deploy` the fix turns green, none listed
means every target; --pr is the pull request number.
update <slug> --status staged|pr|merged [--sha <commit>] [--pr <n>]
[--repo <abs clone>] [--default <branch>]: move a hotfix on. --sha
resolves in the recorded git common dir or in --repo, which must be
the same repository unless the recorded one is unknown, and records
it anew. --status merged needs --sha, the commit as it sits on
origin/<default> after a fetch.
pending --repo <dir> [--target <t> ...]: every hotfix of that repository
against HEAD; with --target only those that turn one of the targets
green, as void counts them.
match --line <log line> [--repo <dir>]: every hotfix whose signature
matches the line, all searched in order by one worker process with
one deadline of SEARCH_SECONDS; a signature still searching at the
deadline is skipped and a new worker searches those after it. With
--repo, those of another repository are labelled and do not count.
void --repo <dir> --target <t> [--target ...]: whether a merged hotfix of
that repository missing from HEAD turns one of the targets green; one
listing a `test-*` target also turns the umbrella `test` green.
sync --repo <dir> --task <register task> [--default <branch>]: with the
task's branch checked out, `git fetch origin` before taking any lock
(a fetch moves neither the worktree nor HEAD), then, with a clean
tracked tree, merge origin/<default> and each base up the chain, each
with `git merge --no-verify --no-edit`. From the tree check to the
last merge or abort it holds `kvb-gate-sync.lock`, with its PID and
command recorded there, and the worktree lock of the kvb-check-gate
skill's gate.py, and refuses while a `gate.py run` or `gate.py hold`
holds it, naming the holder. A watch.py busy process it sees in <dir>
also refuses, but processes in another sandbox's PID namespace stay
unseen, so the lock is the guard. The default branch is origin/HEAD; --default is read
only when that ref is missing and must agree with it otherwise.
Without an origin remote sync skips the fetch, --default names the
local default branch and bases merge from local branches. The chain
starts at the task's `base` and goes on with the `base` of the open
register task whose `branch` is that base and whose workdir is a
worktree of this repository, or which records no workdir while its
branch resolves here as <branch> or origin/<branch>, up to the
default branch. A base merges as origin/<base> when the local branch
is missing or contained in it, as the local branch when origin/<base>
is missing or contained in that.
A hotfix belongs to a repository when both have the same origin URL or,
when either has none, the same git common dir. Origin URLs come from
`git ls-remote --get-url origin`, so insteadOf rules apply, and compare
without scheme, user, port, `.git` and trailing slash; a local path
resolves against the worktree's top level, and the path on a
case-insensitive host of CASELESS compares in lowercase. A merged hotfix
missing from HEAD counts only when its sha is on origin/<default>; when
it is not, one fetch decides. Every fetch (sync, pending, void, and add
or update with --status merged) runs with GIT_TERMINAL_PROMPT=0 and is
stopped after FETCH_SECONDS.
Output:
add, update: `added hotfix/<slug>` or `updated hotfix/<slug>`.
pending: `<verdict>\t<task>\t<status>\t<branch>\t<sha>\t<pr>` for each
merged hotfix missing from HEAD (`missing (merge main)`, or
`not on origin/<default>`) and each staged or pr one
(`open, not merged`), then `hotfix: <n> missing, <m> open`.
match: `<task>\t<status>\t<branch>\t<sha>\t<pr>\t<repository>` per match,
with --repo prefixed `foreign\t` for another repository and
`unknown\t` for an unresolvable one, or a `hotfix: no signature`
line when nothing of the repository matches. A signature still
searching at the SEARCH_SECONDS deadline is named on stderr and
counts as no match.
void: `void\t<task>\t<sha>\t<targets>` per hotfix that voids the run; a
hotfix off origin/<default> voids nothing and is named on stderr.
sync: `merged\t<ref>\t<new HEAD>` or `contained\t<ref>` per ref in merge
order; `conflict\t<ref>\t<files>` on the first conflict.
pending and void name every row whose repository is unknown (no origin
URL and its git common dir gone) on stderr; such a row changes no exit
code.
Exit codes:
0 done; pending: none missing; match: a match of the repository; void:
not void.
1 pending: a merged hotfix is missing; match: no match.
2 usage error, unknown task, dirty tracked tree, HEAD not on the task's
branch, a held worktree, a chain row without workdir whose branch
does not resolve, an unresolvable or diverged ref, a merge that
failed without a conflict (aborted), a merged sha off
origin/<default>; pending and void: a missing origin/HEAD or a
failed fetch, unless void exits 3; match: the search worker did not
start, did not get ready before the deadline or exited non-zero.
3 void: a merged hotfix missing from HEAD turns a target green.
4 sync: a merge conflicted and was aborted; earlier merges stay.
"""
from __future__ import annotations
import argparse
import contextlib
import fcntl
import importlib.util
import itertools
import json
import os
import re
import subprocess
import sys
from collections.abc import Callable, Iterator
from pathlib import Path
from types import ModuleType
from typing import NoReturn
SKILLS = Path(__file__).resolve().parents[2]
RASCI = SKILLS / "kvb-org-rasci" / "scripts" / "rasci.py"
GATE = SKILLS / "kvb-check-gate" / "scripts" / "gate.py"
WATCH = SKILLS / "kvb-org-organize" / "scripts" / "watch.py"
PREFIX = "hotfix/"
STAGES = ("staged", "pr", "merged")
REGISTER_STATUS = {"staged": "running", "pr": "running", "merged": "done"}
SHA = re.compile(r"^[0-9a-f]{7,40}$")
TARGET = re.compile(r"^[A-Za-z0-9_][A-Za-z0-9._%/+-]*$")
UMBRELLA = "test"
ORIGIN_HEAD = "refs/remotes/origin/HEAD"
GIT = ("git", "-c", "maintenance.auto=false")
FETCH = ("fetch", "--quiet", "--no-auto-maintenance", "origin")
FETCH_SECONDS = 120
URL = re.compile(r"^[A-Za-z][A-Za-z0-9+.-]*://(?:[^@/]*@)?(?P<host>[^/:]*)(?::\d*)?(?P<path>/.*)?$")
SCP = re.compile(r"^(?:[^@/:]+@)?(?P<host>[^/:]+):(?P<path>.*)$")
CASELESS = ("github.com", "gitlab.com", "codeberg.org", "bitbucket.org")
SEARCH_SECONDS = 3
SEARCH = (
"import json, re, sys\n"
"data = json.loads(sys.stdin.buffer.read())\n"
"print('ready', flush=True)\n"
"for pattern in data['patterns']:\n"
" print(int(re.search(pattern, data['line']) is not None), flush=True)\n"
)
CLASSES = {"DIGIT": r"\d", "SPACE": r"\s", "WORD": r"\w", "LINEBREAK": r"\n"}
PROBES = frozenset(map(chr, range(128))) | {"é", "٠", "　"}
NESTED = "repeats a group that repeats itself, like (a+)+"
OVERLAP = "repeats an alternation whose branches can start alike, like (a|ab|b)*"
def fail(message: str) -> NoReturn:
print(f"hotfix: {message}", file=sys.stderr)
sys.exit(2)
def report(problems: list[str]) -> None:
for problem in problems:
print(f"hotfix: {problem}", file=sys.stderr)
def module(name: str, path: Path) -> ModuleType:
spec = importlib.util.spec_from_file_location(name, path)
loaded = importlib.util.module_from_spec(spec)
spec.loader.exec_module(loaded)
return loaded
def git(where: str, *args: str) -> subprocess.CompletedProcess:
try:
return subprocess.run(
[*GIT, *args], cwd=where, stdin=subprocess.DEVNULL, capture_output=True, text=True, check=False
)
except OSError as error:
return subprocess.CompletedProcess([*GIT, *args], 128, "", str(error))
def git_out(where: str, *args: str) -> str:
result = git(where, *args)
if result.returncode:
fail(f"git {' '.join(args)} in {where}: {(result.stderr or result.stdout).strip()}")
return result.stdout
def fetch(where: str) -> subprocess.CompletedProcess:
command = [*GIT, *FETCH]
try:
return subprocess.run(
command, cwd=where, stdin=subprocess.DEVNULL, capture_output=True, text=True,
env={**os.environ, "GIT_TERMINAL_PROMPT": "0"}, timeout=FETCH_SECONDS, check=False,
)
except subprocess.TimeoutExpired:
return subprocess.CompletedProcess(command, 124, "", f"stopped after {FETCH_SECONDS} s")
except OSError as error:
return subprocess.CompletedProcess(command, 128, "", str(error))
def fetched(where: str) -> None:
result = fetch(where)
if result.returncode:
fail(f"git fetch origin in {where}: {(result.stderr or result.stdout).strip()}")
def ancestor(where: str, rev: str, of: str) -> bool:
return git(where, "merge-base", "--is-ancestor", rev, of).returncode == 0
def resolves(where: str, ref: str) -> bool:
return git(where, "rev-parse", "--verify", "--quiet", f"{ref}^{{commit}}").returncode == 0
def task_of(rasci: ModuleType, slug: str) -> str:
if not rasci.TASK.match(slug):
fail(f"invalid slug {slug!r}; use lowercase letters, digits, '.', '_', '-' and '/'")
return PREFIX + slug
def worktree(value: str) -> str:
path = os.path.realpath(value)
if not os.path.isdir(path):
fail(f"{value} is no directory")
if identity(path) is None:
fail(f"{value} is not a git repository")
return path
def clone(value: str) -> str:
if not os.path.isabs(value):
fail(f"--repo {value!r} is not an absolute path")
return worktree(value)
def normalized(url: str, top: str) -> str:
"""Compare form of an origin URL.
Args:
url: the origin URL with insteadOf rules applied.
top: the directory a local relative path is relative to.
"""
url = url.strip()
found = URL.match(url) or (None if url.startswith((".", "/")) else SCP.match(url))
host, path = (found["host"].lower(), found["path"] or "") if found else ("", url)
if host:
path = path.rstrip("/").removesuffix(".git").rstrip("/").lstrip("/")
return f"{host}/{path.lower() if host in CASELESS else path}"
if not path:
return ""
return os.path.realpath(os.path.join(top, path)).removesuffix(".git").rstrip("/")
def origin_url(where: str) -> str:
if not git(where, "config", "--get", "remote.origin.url").stdout.strip():
return ""
url = git(where, "ls-remote", "--get-url", "origin").stdout
top = git(where, "rev-parse", "--show-toplevel")
return normalized(url, top.stdout.strip() if top.returncode == 0 else where)
def identity(where: str) -> tuple[str, str] | None:
if not where or not os.path.isdir(where):
return None
common = git(where, "rev-parse", "--path-format=absolute", "--git-common-dir")
if common.returncode:
return None
return os.path.realpath(common.stdout.strip()), origin_url(where)
def same(one: tuple[str, str], other: tuple[str, str]) -> bool:
if one[1] and other[1]:
return one[1] == other[1]
return one[0] == other[0]
def recorded(row: dict) -> tuple[str, str] | None:
common, origin = row.get("common_dir") or "", row.get("origin") or ""
if not origin and not os.path.isdir(common):
return None
return common, origin
def repository(row: dict) -> str:
return row.get("origin") or row.get("common_dir") or "-"
def unknown(task: str, row: dict) -> str:
return (
f"{task}: repository unknown (no origin URL, git dir {row.get('common_dir') or '-'} gone); record it with "
f"`hotfix.py update {task.removeprefix(PREFIX)} --status {row['stage']} --sha {row['sha']} --repo <clone>`"
)
def hex_id(sha: str) -> str:
if not SHA.match(sha):
fail(f"--sha {sha!r} is no hex commit id of 7 to 40 characters")
return sha
def commit(where: str, sha: str) -> str:
found = git(where, "rev-parse", "--verify", "--quiet", f"{hex_id(sha)}^{{commit}}")
if found.returncode:
fail(f"--sha {sha} is no commit in {where}")
return found.stdout.strip()
def default_branch(where: str, given: str | None) -> str:
head = git(where, "symbolic-ref", "--quiet", ORIGIN_HEAD)
if head.returncode == 0:
name = head.stdout.strip().removeprefix("refs/remotes/origin/")
if given is not None and given != name:
fail(f"origin/HEAD names {name}, --default says {given}")
return name
if given is None:
fail(f"{ORIGIN_HEAD} is missing in {where}; pass --default <branch> or run `git remote set-head origin --auto`")
return branch_name(where, given, "--default")
def landed(where: str, sha: str, given: str | None) -> str:
hex_id(sha)
fetched(where)
name = default_branch(where, given)
full = commit(where, sha)
if not ancestor(where, full, f"refs/remotes/origin/{name}"):
fail(f"--sha {sha} is not on origin/{name}; record the commit as it sits there, a squash or rebase merge "
"changes it")
return full
def branch_name(where: str, name: str, what: str) -> str:
if name.startswith("-") or git(where, "check-ref-format", f"refs/heads/{name}").returncode:
fail(f"{what} {name!r} is no valid branch name")
return name
def children(value: object, subpattern: type) -> list:
if isinstance(value, subpattern):
return [value]
if isinstance(value, (tuple, list)):
return [sub for item in value for sub in children(item, subpattern)]
return []
def either(tests: list[Callable[[str], bool]]) -> Callable[[str], bool]:
return lambda char: any(test(char) for test in tests)
def risk(text: str) -> str | None:
parser, c = getattr(re, "_parser", None), getattr(re, "_constants", None)
if parser is None or c is None:
print("hotfix: this Python hides re's parser; --signature is not checked for runaway backtracking",
file=sys.stderr)
return None
loose = (c.MAX_REPEAT, c.MIN_REPEAT)
repeats = (*loose, c.POSSESSIVE_REPEAT)
barriers = (c.ATOMIC_GROUP, c.POSSESSIVE_REPEAT)
probes = PROBES | set(text)
def parts(av: object) -> list:
return children(av, parser.SubPattern)
def unbounded(op: object, av: object) -> bool:
return op in loose and av[1] == c.MAXREPEAT
def inside(pattern: object, hit: Callable[[object, object], bool]) -> bool:
return any(
hit(op, av) or (op not in barriers and any(inside(part, hit) for part in parts(av)))
for op, av in pattern.data
)
def overlapping(op: object, av: object) -> bool:
if op != c.BRANCH:
return False
heads = [start(branch) for branch in av[1]]
return any(
(one_empty and two_empty) or any(one(char) and two(char) for char in probes)
for (one, one_empty), (two, two_empty) in itertools.combinations(heads, 2)
)
def start(pattern: object) -> tuple[Callable[[str], bool], bool]:
tests = []
for op, av in pattern.data:
test, empty = first(op, av)
tests.append(test)
if not empty:
return either(tests), False
return either(tests), True
def first(op: object, av: object) -> tuple[Callable[[str], bool], bool]:
if op == c.LITERAL:
return (lambda char: char == chr(av)), False
if op == c.NOT_LITERAL:
return (lambda char: char != chr(av)), False
if op == c.ANY:
return (lambda char: True), False
if op == c.IN:
return member(av), False
if op in (c.AT, c.ASSERT, c.ASSERT_NOT):
return (lambda char: False), True
if op in repeats:
test, empty = start(av[2])
return test, empty or av[0] == 0
if op in (c.SUBPATTERN, c.ATOMIC_GROUP, c.BRANCH):
heads = [start(part) for part in parts(av)]
return either([test for test, _ in heads]), any(empty for _, empty in heads)
return (lambda char: True), True
def member(items: list) -> Callable[[str], bool]:
negated = bool(items) and items[0][0] == c.NEGATE
kept = [item for item in items if item[0] != c.NEGATE]
return lambda char: any(holds(op, av, char) for op, av in kept) != negated
def holds(op: object, av: object, char: str) -> bool:
if op == c.LITERAL:
return char == chr(av)
if op == c.RANGE:
return av[0] <= ord(char) <= av[1]
if op == c.CATEGORY:
name = av.name.removeprefix("CATEGORY_").removeprefix("LOC_").removeprefix("UNI_")
klass = CLASSES.get(name.removeprefix("NOT_"))
return klass is None or (re.fullmatch(klass, char) is not None) != name.startswith("NOT_")
return True
def scan(pattern: object) -> str | None:
for op, av in pattern.data:
if unbounded(op, av) and inside(av[2], unbounded):
return NESTED
if unbounded(op, av) and inside(av[2], overlapping):
return OVERLAP
for part in parts(av):
found = scan(part)
if found is not None:
return found
return None
return scan(parser.parse(text))
def signature(text: str) -> str:
if not text.strip():
fail("--signature must not be empty")
try:
re.compile(text)
except re.error as error:
fail(f"--signature {text!r}: {error}")
found = risk(text)
if found is not None:
fail(f"--signature {text!r} {found}, which backtracks without end on a near miss; match the stable part "
"instead")
return text
def searched(patterns: list[str], line: str) -> list[bool]:
"""Search the line with every pattern in order in one worker until SEARCH_SECONDS pass.
Args:
patterns: signatures that compile.
line: the log line.
Returns:
one verdict per pattern; fewer when the deadline passed, the next pattern still searching.
"""
payload = json.dumps({"line": line, "patterns": patterns}).encode("ascii")
late = False
try:
with subprocess.Popen(
[sys.executable, "-I", "-c", SEARCH], stdin=subprocess.PIPE, stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
) as worker:
try:
out, err = worker.communicate(payload, timeout=SEARCH_SECONDS)
except subprocess.TimeoutExpired:
late = True
worker.kill()
out, err = worker.communicate()
except OSError as error:
fail(f"cannot start the signature search worker {sys.executable}: {error.strerror or error}")
*lines, tail = out.decode("ascii", "replace").split("\n")
verdicts = lines[1:]
if late and lines[:1] != ["ready"]:
fail(f"the signature search worker did not get ready within {SEARCH_SECONDS} s")
complete = late or (worker.returncode == 0 and not tail and len(verdicts) == len(patterns))
if lines[:1] != ["ready"] or not complete or len(verdicts) > len(patterns) or set(verdicts) - {"0", "1"}:
reason = err.decode("utf-8", "replace").strip().splitlines()[-1:] or ["no message"]
fail(f"the signature search worker exited {worker.returncode}: {reason[0]}")
return [verdict == "1" for verdict in verdicts]
def make_targets(values: list[str] | None) -> list[str]:
for value in values or []:
if not TARGET.match(value):
fail(f"--target {value!r} is no make target name")
return list(dict.fromkeys(values or []))
def turns_green(greens: list[str], wanted: set[str]) -> bool:
if not greens or wanted & set(greens):
return True
return UMBRELLA in wanted and any(green.startswith(f"{UMBRELLA}-") for green in greens)
def number(value: int | None) -> int | None:
if value is not None and value < 1:
fail("--pr must be 1 or higher")
return value
def hotfixes(rasci: ModuleType) -> list[tuple[str, dict]]:
return [(t, row) for t, row in sorted(rasci.load().items()) if t.startswith(PREFIX) and "stage" in row]
def of_repository(rasci: ModuleType, repo: str) -> tuple[list[tuple[str, dict]], list[str]]:
here = identity(repo)
rows, unknowns = [], []
for task, row in hotfixes(rasci):
theirs = recorded(row)
if theirs is None:
unknowns.append(unknown(task, row))
elif same(theirs, here):
rows.append((task, row))
return rows, unknowns
def for_targets(rows: list[tuple[str, dict]], targets: list[str] | None) -> list[tuple[str, dict]]:
if targets is None:
return rows
wanted = set(make_targets(targets))
return [(task, row) for task, row in rows if turns_green(row.get("targets") or [], wanted)]
def missing(repo: str, rows: list[tuple[str, dict]], problems: list[str]) -> tuple[list, list, str]:
absent = [(t, row) for t, row in rows if row["stage"] == "merged" and not ancestor(repo, row["sha"], "HEAD")]
if not absent:
return [], [], ""
head = git(repo, "symbolic-ref", "--quiet", ORIGIN_HEAD)
if head.returncode:
problems.append(f"{ORIGIN_HEAD} is missing in {repo}, so {', '.join(t for t, _ in absent)} stay unchecked; "
"run `git remote set-head origin --auto`")
return [], [], ""
ref = head.stdout.strip()
on = {t for t, row in absent if ancestor(repo, row["sha"], ref)}
if len(on) < len(absent):
result = fetch(repo)
if result.returncode:
unchecked = ", ".join(t for t, _ in absent if t not in on)
problems.append(f"git fetch origin in {repo} failed, so {unchecked} stay unchecked: {result.stderr.strip()}")
return [(t, row) for t, row in absent if t in on], [], ""
on = {t for t, row in absent if ancestor(repo, row["sha"], ref)}
shown = ref.removeprefix("refs/remotes/")
return [(t, row) for t, row in absent if t in on], [(t, row) for t, row in absent if t not in on], shown
def pr_cell(row: dict) -> str:
return f"#{row['pr']}" if row.get("pr") else "-"
def cmd_add(rasci: ModuleType, args: argparse.Namespace) -> int:
task = task_of(rasci, args.slug)
if task in rasci.load():
fail(f"{task} is already in the register; move it on with update")
repo = clone(args.repo)
common, origin = identity(repo)
entry = {
"common_dir": common,
"origin": origin,
"branch": branch_name(repo, args.branch, "--branch"),
"signature": signature(args.signature),
"targets": make_targets(args.target),
"stage": args.status,
}
pr = number(args.pr)
if pr is not None:
entry["pr"] = pr
entry["sha"] = landed(repo, args.sha, args.default) if args.status == "merged" else commit(repo, args.sha)
council = module("council", rasci.COUNCIL)
sid, _ = council.caller(args.session)
entry["roles"] = {council.live_sessions().get(sid) or sid: "RA"}
rasci.apply_status(entry, REGISTER_STATUS[args.status])
with rasci.database() as tasks:
if task in tasks:
fail(f"{task} is already in the register; move it on with update")
tasks[task] = entry
print(f"added {task}")
return 0
def located(task: str, entry: dict, given: str | None) -> tuple[str, dict]:
if given is None:
common = entry.get("common_dir") or ""
if not os.path.isdir(common):
fail(f"{task}: its git dir {common or '-'} is gone; pass --repo <a clone of {repository(entry)}>")
return common, {}
repo = clone(given)
here = identity(repo)
theirs = recorded(entry)
if theirs is not None and not same(theirs, here):
fail(f"--repo {given} is another repository than {task}'s {repository(entry)}")
return repo, {"common_dir": here[0], "origin": here[1]}
def cmd_update(rasci: ModuleType, args: argparse.Namespace) -> int:
task = task_of(rasci, args.slug)
pr = number(args.pr)
if args.status == "merged" and args.sha is None:
fail("--status merged needs --sha, the commit as it sits on the default branch")
entry = rasci.load().get(task)
if entry is None or "stage" not in entry:
fail(f"{task}: no hotfix in the register; record it with add")
fields: dict = {}
if args.repo is not None or args.sha is not None:
where, fields = located(task, entry, args.repo)
if args.sha is not None:
fields["sha"] = landed(where, args.sha, args.default) if args.status == "merged" else commit(where, args.sha)
if pr is not None:
fields["pr"] = pr
with rasci.database() as tasks:
entry = tasks.get(task)
if entry is None or "stage" not in entry:
fail(f"{task}: no hotfix in the register; record it with add")
entry.update(fields)
entry["stage"] = args.status
rasci.apply_status(entry, REGISTER_STATUS[args.status])
print(f"updated {task}")
return 0
def cmd_pending(rasci: ModuleType, args: argparse.Namespace) -> int:
repo = worktree(args.repo)
rows, unknowns = of_repository(rasci, repo)
rows = for_targets(rows, args.target)
problems: list[str] = []
on, off, shown = missing(repo, rows, problems)
verdicts = {t: "missing (merge main)" for t, _ in on} | {t: f"not on {shown}" for t, _ in off}
opened = 0
for task, row in rows:
if row["stage"] != "merged":
verdict = "open, not merged"
opened += 1
elif task in verdicts:
verdict = verdicts[task]
else:
continue
print("\t".join((verdict, task, row["stage"], row["branch"], row["sha"], pr_cell(row))))
report([*unknowns, *problems])
print(f"hotfix: {len(on)} missing, {opened} open")
if problems:
return 2
return 1 if on else 0
def cmd_match(rasci: ModuleType, args: argparse.Namespace) -> int:
here = identity(worktree(args.repo)) if args.repo is not None else None
rows = []
for task, row in hotfixes(rasci):
try:
re.compile(row["signature"])
except re.error as error:
print(f"hotfix: {task}: signature does not compile ({error}), skipped", file=sys.stderr)
continue
rows.append((task, row))
hits = []
while rows:
verdicts = searched([row["signature"] for _, row in rows], args.line)
hits += [entry for entry, hit in zip(rows[:len(verdicts)], verdicts, strict=True) if hit]
if len(verdicts) == len(rows):
break
task, row = rows[len(verdicts)]
print(f"hotfix: {task}: signature {row['signature']!r} still searched this line at the {SEARCH_SECONDS} s "
"deadline, skipped; its A must narrow it", file=sys.stderr)
rows = rows[len(verdicts) + 1:]
found = 0
for task, row in hits:
cells = [task, row["stage"], row["branch"], row["sha"], pr_cell(row), repository(row)]
theirs = recorded(row)
if here is not None and theirs is None:
cells.insert(0, "unknown")
elif here is not None and not same(theirs, here):
cells.insert(0, "foreign")
else:
found += 1
print("\t".join(cells))
if not found:
print("hotfix: no signature matches" if here is None else "hotfix: no signature of this repository matches")
return 0 if found else 1
def cmd_void(rasci: ModuleType, args: argparse.Namespace) -> int:
repo = worktree(args.repo)
rows, unknowns = of_repository(rasci, repo)
rows = for_targets(rows, args.target)
problems: list[str] = []
on, off, shown = missing(repo, rows, problems)
for task, row in off:
print(f"hotfix: {task}: {row['sha']} is not on {shown} (a squash or rebase merge?), so it voids nothing; "
"its A must record the commit as it sits there", file=sys.stderr)
for task, row in on:
print("\t".join(("void", task, row["sha"], ",".join(row.get("targets") or []) or "all targets")))
report([*unknowns, *problems])
if on:
return 3
return 2 if problems else 0
@contextlib.contextmanager
def locked(repo: str, command: list[str]) -> Iterator[None]:
"""Hold gate.py's sync lock and worktree lock of the worktree.
Args:
repo: the worktree.
command: sync's command line, recorded in the sync lock.
"""
if not GATE.is_file():
yield
return
gate = module("gate", GATE)
git_dir = Path(git_out(repo, "rev-parse", "--absolute-git-dir").strip())
with (
open(git_dir / gate.SYNC_LOCK, "a", encoding="utf-8") as sync,
open(git_dir / gate.RUN_LOCK, "a", encoding="utf-8") as run,
):
fcntl.flock(sync, fcntl.LOCK_EX)
gate.claim(sync, command)
try:
fcntl.flock(run, fcntl.LOCK_EX | fcntl.LOCK_NB)
except BlockingIOError:
fail(f"a gate.py run or hold holds {repo} ({gate.holder(git_dir / gate.RUN_LOCK)}); sync once it ends")
yield
def watched(repo: str) -> None:
if WATCH.is_file():
watch = module("watch", WATCH)
config = watch.load_config()
if config is not None and watch.busy(repo, config["busy_processes"]):
fail(f"a busy process (watch.py busy_processes) runs in {repo}, a gate or deploy; sync once it ends")
def chain(repo: str, tasks: dict[str, dict], task: str, default: str, ended: tuple[str, ...]) -> list[str]:
here = identity(repo)
places: dict[str, tuple[str, str] | None] = {}
found: list[str] = []
base = tasks[task].get("base")
while base and base != default:
if base in found:
fail(f"base chain of {task} loops: {' <- '.join([*found, base])}")
found.append(branch_name(repo, base, "base"))
upstream = {}
for name, row in sorted(tasks.items()):
if row.get("branch") != base or not row.get("base") or row.get("status") in ended:
continue
workdir = row.get("workdir") or ""
if not workdir:
if not (resolves(repo, f"refs/heads/{base}") or resolves(repo, f"refs/remotes/origin/{base}")):
fail(f"{name} records no workdir and its branch {base} resolves neither as {base} nor as "
f"origin/{base} in {repo}; record it with `rasci.py set {name} --workdir <dir>`")
upstream[name] = row["base"]
continue
if workdir not in places:
places[workdir] = identity(workdir)
if places[workdir] is None:
print(f"hotfix: {name}: workdir {workdir} is no git repository, its base is not followed",
file=sys.stderr)
elif same(places[workdir], here):
upstream[name] = row["base"]
if len(set(upstream.values())) > 1:
named = ", ".join(f"{t}={b}" for t, b in sorted(upstream.items()))
fail(f"branch {base} has several bases in the register: {named}")
base = next(iter(upstream.values()), None)
return found
def base_ref(repo: str, base: str) -> tuple[str, str]:
local, remote = f"refs/heads/{base}", f"refs/remotes/origin/{base}"
if not resolves(repo, remote):
if not resolves(repo, local):
fail(f"{base} does not resolve in {repo}")
return base, local
if not resolves(repo, local) or ancestor(repo, local, remote):
return f"origin/{base}", remote
if ancestor(repo, remote, local):
return base, local
fail(f"{base} and origin/{base} have diverged in {repo}; merge one into the other first")
def default_ref(repo: str, given: str | None) -> tuple[str, tuple[str, str]]:
if git(repo, "config", "--get", "remote.origin.url").stdout.strip():
default = default_branch(repo, given)
fetched(repo)
return default, (f"origin/{default}", f"refs/remotes/origin/{default}")
if given is None:
fail(f"{repo} has no origin remote; pass --default <branch>, its local default branch")
default = branch_name(repo, given, "--default")
return default, (default, f"refs/heads/{default}")
def cmd_sync(rasci: ModuleType, args: argparse.Namespace) -> int:
repo = worktree(args.repo)
tasks = rasci.load()
if args.task not in tasks:
fail(f"{args.task}: not in the register")
branch = tasks[args.task].get("branch")
if not branch:
fail(f"{args.task} records no branch; set it with `rasci.py set {args.task} --branch <branch>`")
head = git(repo, "symbolic-ref", "--quiet", "HEAD").stdout.strip()
if head != f"refs/heads/{branch}":
fail(f"{repo} has {head.removeprefix('refs/heads/') or 'a detached HEAD'} checked out, "
f"{args.task} works on {branch}")
default, first = default_ref(repo, args.default)
with locked(repo, ["hotfix.py", "sync", "--repo", repo, "--task", args.task]):
watched(repo)
dirty = [line[3:] for line in git_out(repo, "status", "--porcelain=v1", "--untracked-files=no").splitlines()]
if dirty:
fail(f"tracked files are dirty, commit them first: {', '.join(dirty)}")
if not resolves(repo, first[1]):
fail(f"{first[0]} does not resolve in {repo}")
bases = chain(repo, tasks, args.task, default, rasci.ENDED)
for shown, ref in [first, *(base_ref(repo, base) for base in bases)]:
if ancestor(repo, ref, "HEAD"):
print(f"contained\t{shown}")
continue
result = git(repo, "merge", "--no-verify", "--no-edit", ref)
if result.returncode == 0:
print(f"merged\t{shown}\t{git_out(repo, 'rev-parse', 'HEAD').strip()}")
continue
files = git(repo, "diff", "--name-only", "--diff-filter=U").stdout.splitlines()
if git(repo, "rev-parse", "--verify", "--quiet", "MERGE_HEAD").returncode == 0:
git_out(repo, "merge", "--abort")
if not files:
fail(f"git merge {shown} failed and was aborted: {(result.stderr or result.stdout).strip()}")
print(f"conflict\t{shown}\t{','.join(files)}")
print(f"hotfix: merging {shown} conflicts in {', '.join(files)}; merge aborted", file=sys.stderr)
return 4
return 0
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
sub = parser.add_subparsers(dest="cmd", required=True)
p = sub.add_parser("add")
p.add_argument("slug")
for flag in ("--repo", "--branch", "--sha", "--signature"):
p.add_argument(flag, required=True)
p.add_argument("--status", choices=STAGES, required=True)
p.add_argument("--target", action="append")
p.add_argument("--pr", type=int)
p.add_argument("--default")
p.add_argument("--session")
p.set_defaults(func=cmd_add)
p = sub.add_parser("update")
p.add_argument("slug")
p.add_argument("--status", choices=STAGES, required=True)
p.add_argument("--sha")
p.add_argument("--pr", type=int)
p.add_argument("--repo")
p.add_argument("--default")
p.set_defaults(func=cmd_update)
p = sub.add_parser("pending")
p.add_argument("--repo", required=True)
p.add_argument("--target", action="append")
p.set_defaults(func=cmd_pending)
p = sub.add_parser("match")
p.add_argument("--line", required=True)
p.add_argument("--repo")
p.set_defaults(func=cmd_match)
p = sub.add_parser("void")
p.add_argument("--repo", required=True)
p.add_argument("--target", action="append", required=True)
p.set_defaults(func=cmd_void)
p = sub.add_parser("sync")
p.add_argument("--repo", required=True)
p.add_argument("--task", required=True)
p.add_argument("--default")
p.set_defaults(func=cmd_sync)
args = parser.parse_args()
return args.func(module("rasci", RASCI), args)
if __name__ == "__main__":
sys.exit(main())
skills/kvb-org-coordinate/SKILL.md
+5 -2
View File
@@ -8,5 +8,8 @@ description: >
Run the `kvb-org-organize` skill's **Peer protocol**, its `lineage.md`
included, for the failure or fix at hand. They hold the whole procedure,
including the entries in the `kvb-org-rasci` register: ask by `SendMessage`,
report by `news.py post`. This skill only keeps its command working.
including the entries in the `kvb-org-rasci` register: look first, `hotfix.py
match` of the `kvb-git-hotfix` skill included; ask by `SendMessage`; report
by `news.py post`. A hotfix is recorded with `hotfix.py add` and `update`,
never reported by message or news post. This skill only keeps its command
working.
skills/kvb-org-organize/SKILL.md
+20 -17
View File
@@ -7,7 +7,7 @@ description: >
---
The first claim owns a piece of work; a later agent hands its findings to the
owner and never builds a second version. Offices are recallable mandates
owner, never building a second version. Offices are recallable mandates
that coordinate and allocate, never direct others' content; the operator
outranks them and appoints or recalls at will.
@@ -18,15 +18,15 @@ outranks them and appoints or recalls at will.
1. Name the office: `work/<slug>` (a piece of work), `cluster/<slug>` (a
strand across lines), `<dimension>/<line>` (a line) or `chair`, the slug
concrete (issue number, failing line, log line). The first agent elected
concrete (issue number, failing log line). The first agent elected
while `chair` is vacant also claims it. Read `offices.md` next to this
file before claiming a cluster, a line or the chair.
2. `council.py list`; the same work claimed under another slug counts.
3. `council.py claim <office> --work "<what, evidence, branch or worktree>"`.
Exit 0: yours; 3: a live holder (printed); 4: the holder's session is dead.
Exit 0: yours; 3: a live holder (printed); 4: a dead holder.
4. Elected: complete the task the claim entered in the `kvb-org-rasci`
register (objective, description, priority, roles); the listed claim is
the announcement.
register (objective, description, priority, roles); the listed claim
announces it.
5. Not elected: post your findings to the holder via the news and stop.
A hook (`council.py gate`) blocks worktree edits before a claim. Between
@@ -42,9 +42,11 @@ ends or goes stale, or after a restart.
- `make test`, `make test-lint`, quality gates, deploys, iteration loops,
swarm, image builds, cloud, shared-data cleanup, anything with Docker or
over a few minutes: ask the chair (what, where, how long, disk), wait for
the grant, post the release to the news.
the grant, post the release to the news. In a worktree they run through
`gate.py run` or `gate.py hold` (`kvb-check-gate`).
- No running job is cut short for capacity; only the operator orders one
down. `RESOURCE` hook lines address the chair; others adjust plans.
down. Exception: void by a missing merged hotfix (`kvb-git-hotfix`).
`RESOURCE` hook lines address the chair; others adjust plans.
- Read `capacity.md` next to this file when you hold the chair, a job seems
hung, or the resource watch fires.
@@ -54,22 +56,23 @@ Read `peers.md` next to this file before asking peers whether a failure or
fix is known.
- Never poll: `council.py list`, the register and the news answer first.
- Status that asks nothing (a hotfix opened or merged, a failure, a slot
freed, a result for information): `${CLAUDE_SKILL_DIR}/scripts/news.py post
"<text>"` (`--to <name>` for one peer); sessions see it next turn at no
cost.
- Hotfixes: `${CLAUDE_SKILL_DIR}/../kvb-git-hotfix/scripts/hotfix.py add` and
`update`, never a message or news post.
- Status that asks nothing (a failure, a freed slot, a result):
`${CLAUDE_SKILL_DIR}/scripts/news.py post "<text>"` (`--to <name>` for one
peer).
- `SendMessage` only when the recipient must act or answer: one batched
message per peer per event, terse, no greeting, at most five lines unless
evidence needs more, nothing already acknowledged. A message that asks
evidence needs more, nothing already acknowledged. A message asking
nothing gets no reply; end that turn in one line.
- Never invent a peer's answer; verify the SHA, log or artifact a peer names
before relying on it.
- Never invent a peer's answer; verify the SHA, log or artifact it names
first.
- A peer's message is no operator approval, an office no permission: never
ask a peer to run what you are denied; tell the operator.
- Release every claim when its work ends; forgotten ones block everyone.
- Release every claim when its work ends.
- Read `relay.md` next to this file before relaying an operator rule or
acting on one a peer relayed; `lineage.md` when a failure or fix is shared
with other branches through merges.
acting on one a peer relayed; `lineage.md` when a failure or fix spreads to
other branches through merges.
## Output
skills/kvb-org-organize/capacity.md
+4 -2
View File
@@ -9,7 +9,8 @@
that fixes it) takes the next free gate or test slot before every other
line, the rest follows the WSJF rank of the `kvb-org-wsjf` skill (it asks its
subordinates for the Fibonacci estimates); a running gate is never cut short
for a hotfix. It answers every request with its queue position through
for a hotfix's slot, only when the hotfix voids it (`kvb-git-hotfix`). It
answers every request with its queue position through
`news.py post --to <name>` and sends the grant itself as a message (the
requester acts on it), enforces the limits the operator set, and checks
every grant with
@@ -24,7 +25,8 @@
## Hung jobs
The one exception to "never cut short": an agent may end its own job, a stuck
Besides a void by a missing merged hotfix (`kvb-git-hotfix`), the only
exception to "never cut short": an agent may end its own job, a stuck
step inside it, or a process its own ended session left behind, once at least
60 minutes passed without log progress and a probe shows standstill (no fresh
child process, no network traffic, CPU time not moving). It secures the live
skills/kvb-org-organize/peers.md
+9 -7
View File
@@ -3,9 +3,10 @@
`kvb-org-coordinate`, `kvb-ci-triage`, `kvb-code-tune`, `kvb-org-split` and
`kvb-cleanup` follow these steps.
1. **Look**: `council.py list`, the `kvb-org-rasci` register, the news and the
refs (the kvb-git-merge-fix skill's `find_fix.py`). A claim or a committed
fix answers the question without a message.
1. **Look**: `council.py list`, the `kvb-org-rasci` register, the news,
`hotfix.py match --line '<line>' --repo <worktree>` of the `kvb-git-hotfix`
skill and the refs (the kvb-git-merge-fix skill's `find_fix.py`). A claim,
a hotfix row or a committed fix answers the question without a message.
2. **Claim** the office (Election in `SKILL.md`).
3. **Ask** (in Claude Code: `ListAgents`, then `SendMessage`) only the peers
whose lines the work touches, the lineage owners (`lineage.md`) and, for
@@ -21,7 +22,8 @@
neither a "no" nor a competing claim: after 5 minutes, proceed.
5. **Report** to the same peers through `news.py post --to <name>`: commit,
branch, what changed, the evidence it works, and the merge it reaches them
by; an abandoned or failed attempt too. In the register, enter the peers
told as I and mark every key result the evidence proves as met. Then
release the claim: the task ends `done`, or `stopped` while a key result is
open.
by; an abandoned or failed attempt too. A hotfix is no report: it is
recorded with `hotfix.py add` and `update`, never by message or news post.
In the register, enter the peers told as I and mark every key result the
evidence proves as met. Then release the claim: the task ends `done`, or
`stopped` while a key result is open.
skills/kvb-org-rasci/SKILL.md
+4 -3
View File
@@ -13,7 +13,7 @@ itself (holder as R and A, `running`, start, workdir, branch); its release
sets `done` and the end, or `stopped` while a key result is still open.
```bash
rasci.py set <task> --title <t> --objective <o> --description <d> --prio <n> --status queued|running|blocked|done|stopped --worker "<agent | subagent <type> (<model>)>" --workdir <dir> --branch <b> --artifact <kind>:<ref> --tool <script or command> [--start <iso>] [--end <iso>] [--drop-artifact <a>] [--drop-tool <t>]
rasci.py set <task> --title <t> --objective <o> --description <d> --prio <n> --status queued|running|blocked|done|stopped --worker "<agent | subagent <type> (<model>)>" --workdir <dir> --branch <b> --base <upstream branch> --artifact <kind>:<ref> --tool <script or command> [--start <iso>] [--end <iso>] [--drop-artifact <a>] [--drop-tool <t>]
rasci.py kr <task> add "<measurable result>" | met <n> | open <n> | rm <n>
rasci.py role <task> "<party>" <letters>
rasci.py get <task>
@@ -68,5 +68,6 @@ published). `done` is refused while one is open; a task ended short of them is
`get` instead of asking a peer what it works on. `show --wide` adds OKR,
Artifacts, Start, End and every RASCI letter.
- When invoked, print `show --wide` unchanged; its `gaps:` line names tasks
without exactly one A, without R, or open without a key result. `get`
prints one task in full, key results numbered.
without exactly one A, without R, or open without a key result. `hotfix/`
and `operator/` rows are records, not work tasks: `gaps:`, `wsjf` and
`artifacts` skip them. `get` prints one task in full, key results numbered.
skills/kvb-org-rasci/scripts/rasci.py
+7 -4
View File
@@ -39,13 +39,14 @@ STATUSES = ("queued", "running", "blocked", "done", "stopped")
ENDED = ("done", "stopped")
FIELDS = (
"title", "objective", "description", "prio", "status", "worker", "start", "end", "workdir",
"branch",
"branch", "base",
)
TEXT_FIELDS = ("title", "objective", "description", "worker", "workdir", "branch")
TEXT_FIELDS = ("title", "objective", "description", "worker", "workdir", "branch", "base")
LIST_FIELDS = {"tools": "tool"}
ARTIFACT = re.compile(r"^[a-z]+:\S")
FIBONACCI = (1, 2, 3, 5, 8, 13, 21, 34, 55, 89)
OPERATOR = "Operator"
RECORDS = ("hotfix/", "operator/")
def fail(message: str) -> None:
@@ -256,6 +257,8 @@ def okr(row: dict) -> str:
def gaps(task: str, row: dict) -> list[str]:
if task.startswith(RECORDS):
return []
found = []
roles = row["roles"]
accountable = sum("A" in letters for letters in roles.values())
@@ -388,7 +391,7 @@ def cmd_estimate(args: argparse.Namespace) -> int:
def cmd_wsjf(args: argparse.Namespace) -> int:
rows = merged(args.all)
rows = {task: row for task, row in merged(args.all).items() if not task.startswith(RECORDS)}
if not rows:
print("no tasks")
return 0
@@ -414,7 +417,7 @@ def cmd_artifacts(args: argparse.Namespace) -> int:
rows = merged(True)
for task in sorted(rows):
row = rows[task]
if args.task and task != args.task:
if task.startswith(RECORDS) or (args.task and task != args.task):
continue
if args.ended and row["status"] not in (*ENDED, "stale"):
continue
skills/kvb-org-split/SKILL.md
+6 -5
View File
@@ -39,14 +39,16 @@ in a sidetree until it does, or leaves as a proposal.
never into a message.
2. **Sidetree.** Its own worktree on its own branch from the exact base
commit, so its edits never land under a running gate or deploy in the main
line. Use the host's worktree isolation when it has one.
line. Use the host's worktree isolation if any. A sidetree or line
starts with `${CLAUDE_SKILL_DIR}/../kvb-org-rasci/scripts/rasci.py set
<task> --branch <own> --base <upstream> --workdir <dir>`.
3. **Evidence.** Targeted tests, a test for the new behaviour, and a negative
control: the test fails without the change. Numbers come from the same
source before and after.
4. **Self-assessment.** The agent reports fix confidence and break risk with
their residuals and commits on its branch. Its parent marks proven key
results met in the `kvb-org-rasci` register and records
`--branch` and `--artifact worktree:<path>`.
`--artifact worktree:<path>`.
5. **Independent review.** Before any merge, run the `kvb-think-dialectic`
skill with skeptics who did not write the change, briefed the same way
(exact diff and claim, capped answer). A hole returns the change to stage 3.
@@ -58,9 +60,8 @@ in a sidetree until it does, or leaves as a proposal.
## When the bar is out of reach
When the deciding evidence (a live deploy, a production-like dataset, an
upstream answer) is not available to the agent or the change is inherently
riskier, read `proposal.md` next to this file.
When the deciding evidence is not available to the agent or the change is
inherently riskier, read `proposal.md` next to this file.
## Rules
skills/kvb-org-split/proposal.md
+3 -1
View File
@@ -1,6 +1,8 @@
# Proposal
When the bar is out of reach, the agent stops trying and submits a proposal:
When the bar is out of reach (the deciding evidence, such as a live deploy, a
production-like dataset or an upstream answer, is not available, or the change
is inherently riskier), the agent stops trying and submits a proposal:
- what the change does and why it is worth having;
- the confidence and break risk it reached, with their residuals;
tests/test_gate.py
+291 -1
View File
@@ -1,17 +1,28 @@
"""Exercise gate.py: static `make test*` listing, runs judged per target, and the per-HEAD gate."""
"""Exercise gate.py: static `make test*` listing, runs judged per target, hotfix voids, and the per-HEAD gate."""
from __future__ import annotations
import contextlib
import fcntl
import importlib.util
import io
import json
import os
import shutil
import signal
import subprocess
import sys
import tempfile
import time
import unittest
from pathlib import Path
from typing import ClassVar
REPO_ROOT = Path(__file__).resolve().parents[1]
SCRIPT = REPO_ROOT / "skills" / "kvb-check-gate" / "scripts" / "gate.py"
HOTFIX = REPO_ROOT / "skills" / "kvb-git-hotfix" / "scripts" / "hotfix.py"
RUN_LOCK = "kvb-gate-run.lock"
SYNC_LOCK = "kvb-gate-sync.lock"
GREEN = {
"Makefile": (
"include mk/*.mk\n"
@@ -56,7 +67,28 @@ RED = {
"tracked.txt": "one\n",
}
USAGE = {"Makefile": "test:\n\t@true\nbuild:\n\t@true\n"}
VOID = {"Makefile": ".PHONY: test test-a test-b\ntest:\n\t@echo all\ntest-a:\n\t@echo a\ntest-b:\n\t@echo b\n"}
FAILED = ("--fail-regex", r"FAILED TARGETS: \S")
SYNC_HOLDER = "4242\thotfix.py sync --task t\n"
COUNTING_CHILD = (
"import os, signal, sys, time\n"
"got = []\n"
"signal.signal(signal.SIGINT, lambda signum, frame: got.append(signum))\n"
"with open(sys.argv[1] + '.tmp', 'w', encoding='utf-8') as out:\n"
" out.write(str(os.getpid()))\n"
"os.replace(sys.argv[1] + '.tmp', sys.argv[1])\n"
"while not got:\n"
" time.sleep(0.05)\n"
"time.sleep(0.5)\n"
"sys.exit(10 + len(got))\n"
)
def load_gate() -> object:
spec = importlib.util.spec_from_file_location("gate_under_test", SCRIPT)
gate = importlib.util.module_from_spec(spec)
spec.loader.exec_module(gate)
return gate
class GateCase(unittest.TestCase):
@@ -72,6 +104,7 @@ class GateCase(unittest.TestCase):
self.env = {
"PATH": "/usr/bin:/bin",
"HOME": tmp.name,
"AGENTS_COUNCIL_ROOT": str(self.root / "council"),
"GIT_CONFIG_NOSYSTEM": "1",
"GIT_AUTHOR_NAME": "t",
"GIT_AUTHOR_EMAIL": "t@example.org",
@@ -199,6 +232,263 @@ class TestRedPath(GateCase):
self.assertEqual(rows["test"]["status"], "missing")
class TestHotfixes(GateCase):
files = VOID
def setUp(self):
super().setUp()
origin = self.root / "origin.git"
self._git("init", "-q", "--bare", "-b", "main", str(origin))
self._git("remote", "add", "origin", str(origin))
self._git("push", "-q", "-u", "origin", "main")
self._git("remote", "set-head", "origin", "main")
def hotfix(self, slug: str, status: str, *targets: str) -> str:
parent = "refs/remotes/origin/main" if status == "merged" else "HEAD"
sha = self._git("commit-tree", "HEAD^{tree}", "-p", parent, "-m", slug).strip()
if status == "merged":
self._git("push", "-q", "origin", f"{sha}:refs/heads/main")
flags = [part for target in targets for part in ("--target", target)]
result = subprocess.run(
[sys.executable, str(HOTFIX), "add", slug, "--repo", str(self.repo), "--branch", f"fix/{slug}",
"--sha", sha, "--signature", "boom", "--status", status, "--session", "t", *flags],
env=self.env, capture_output=True, text=True, check=False,
)
self.assertEqual(result.returncode, 0, result.stderr)
return sha
def results(self) -> dict[str, str]:
return {r["target"]: r["status"] for r in self.status(self.repo)[1]["targets"]}
def test_a_hotfix_for_any_remaining_target_voids_the_run_before_it_starts(self):
self.hotfix("lint", "pr")
self.hotfix("b", "merged", "test-b")
void = "void: merge main for hotfix b, then rerun\n"
result = self.run_targets("test-a", "test-b", extra=())
self.assertEqual((result.returncode, result.stdout), (3, void), result.stderr)
self.assertEqual(self.results(), {"test": "missing", "test-a": "missing", "test-b": "missing"})
self.assertEqual(self.run_targets("test-a", extra=()).returncode, 0)
self.assertEqual((self.run_targets("test", extra=()).returncode, self.results()["test"]), (3, "missing"))
self._git("merge", "-q", "--ff-only", "origin/main")
self.assertEqual(self.run_targets("test", "test-a", "test-b", extra=()).returncode, 0)
self.hotfix("any", "merged")
result = self.run_targets("test-a", extra=())
self.assertEqual((result.returncode, result.stdout), (3, "void: merge main for hotfix any, then rerun\n"))
def test_status_keeps_the_gate_open_while_a_merged_hotfix_is_missing(self):
self.assertEqual(self.run_targets("test", "test-a", "test-b", extra=()).returncode, 0)
self.hotfix("lint", "pr", "test-a")
self.assertEqual(self.last_status_line(), "gate: green")
self.hotfix("rollout", "merged", "deploy")
self.assertEqual(self.last_status_line(), "gate: green")
self.hotfix("db", "merged", "test-a")
code, status = self.status(self.repo)
self.assertEqual((code, status["gate"], status["open"]), (1, "open", ["missing hotfix: db"]))
self.assertEqual(self.last_status_line(), "gate: open (missing hotfix: db)")
def test_a_failing_or_missing_hotfix_check_warns_and_the_run_goes_on(self):
register = self.root / "council" / "rasci.json"
register.parent.mkdir()
legacy = {"stage": "merged", "repo": str(self.root / "gone"), "branch": "fix/legacy", "sha": "0" * 40,
"signature": "boom", "targets": [], "status": "done", "roles": {}}
register.write_text(json.dumps({"hotfix/legacy": legacy}), encoding="utf-8")
result = self.run_targets("test-a", extra=())
self.assertEqual(result.returncode, 0, result.stderr)
self.assertIn("hotfix/legacy: repository unknown", result.stderr)
self.assertNotIn("going on without its check", result.stderr)
register.write_text("{", encoding="utf-8")
result = self.run_targets("test-a", extra=())
self.assertEqual(result.returncode, 0, result.stderr)
self.assertIn("gate: hotfix.py void exited 2; going on without its check", result.stderr)
status = self.gate("status", "--repo", str(self.repo))
self.assertIn("gate: hotfix.py pending exited 2; going on without its check", status.stderr)
skills = self.root / "lone" / "skills"
shutil.copytree(SCRIPT.parents[1], skills / "kvb-check-gate", ignore=shutil.ignore_patterns("__pycache__"))
lone = subprocess.run(
[sys.executable, str(skills / "kvb-check-gate" / "scripts" / "gate.py"), "run", "--repo", str(self.repo),
"--target", "test-b", "--log-dir", str(self.logs)],
env=self.env, capture_output=True, text=True, check=False,
)
self.assertEqual(lone.returncode, 0, lone.stderr)
self.assertIn("kvb-git-hotfix/scripts/hotfix.py is missing; merged hotfixes are not checked", lone.stderr)
self.assertEqual(self.results()["test-b"], "green")
def test_verdict_lines_count_even_when_hotfix_exits_non_zero(self):
gate = load_gate()
stub = self.root / "stub.py"
gate.HOTFIX = stub
for printed, slugs, warned in (("void\thotfix/x\tabc\tall targets\n", ["x"], False), ("", [], True)):
with self.subTest(printed=printed):
stub.write_text(f"import sys\nsys.stdout.write({printed!r})\nsys.exit(2)\n", encoding="utf-8")
with contextlib.redirect_stderr(io.StringIO()) as err:
self.assertEqual(gate.voiding_hotfixes(self.repo, ["test-a"]), slugs)
self.assertEqual("hotfix.py void exited 2; going on without its check" in err.getvalue(), warned)
class TestWorktreeLock(GateCase):
files = VOID
def lock(self, name: str) -> Path:
return Path(self._git("rev-parse", "--absolute-git-dir").strip()) / name
def lock_free(self) -> bool:
with open(self.lock(RUN_LOCK), "a", encoding="utf-8") as handle:
try:
fcntl.flock(handle, fcntl.LOCK_EX | fcntl.LOCK_NB)
except BlockingIOError:
return False
return True
def test_a_busy_worktree_makes_run_and_hold_exit_75_naming_the_holder(self):
with open(self.lock(RUN_LOCK), "a", encoding="utf-8") as lock:
fcntl.flock(lock, fcntl.LOCK_EX)
lock.write("4242\tgate.py hold -- make deploy\n")
lock.flush()
run = self.run_targets("test-a", extra=())
hold = self.gate("hold", "--repo", str(self.repo), "--", "true")
for result in (run, hold):
with self.subTest(command=result.args[2]):
self.assertEqual(result.returncode, 75, result.stderr)
self.assertIn(f"another gate.py run or hold holds {self.repo} (pid 4242: gate.py hold -- make deploy)",
result.stderr)
self.assertEqual(self.run_targets("test-a", extra=()).returncode, 0)
self.assertIn(f"\tgate.py run --repo {self.repo} --target test-a\n",
self.lock(RUN_LOCK).read_text(encoding="utf-8"))
def test_hold_passes_the_commands_own_exit_codes_through(self):
for code in (1, 2, 3, 75):
with self.subTest(code=code):
result = self.gate("hold", "--repo", str(self.repo), "--", "sh", "-c", f"exit {code}")
self.assertEqual(result.returncode, code)
def test_hold_forwards_signals_to_the_command_and_waits_for_it(self):
for name, signum, group, code in (
("SIGINT to the process group, as Ctrl-C sends it", signal.SIGINT, True, 11),
("SIGTERM to hold only", signal.SIGTERM, False, 128 + signal.SIGTERM),
("SIGHUP to hold only", signal.SIGHUP, False, 128 + signal.SIGHUP),
):
with self.subTest(name):
pid_file = self.root / f"child-{signum}.pid"
holder = subprocess.Popen(
[sys.executable, str(SCRIPT), "hold", "--repo", str(self.repo), "--",
sys.executable, "-c", COUNTING_CHILD, str(pid_file)],
env=self.env, stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL, stderr=subprocess.PIPE,
text=True, start_new_session=True,
)
self.addCleanup(holder.stderr.close)
self.addCleanup(holder.wait)
self.addCleanup(holder.kill)
deadline = time.monotonic() + 30
while not pid_file.exists():
self.assertIsNone(holder.poll())
self.assertLess(time.monotonic(), deadline)
time.sleep(0.05)
child = int(pid_file.read_text(encoding="utf-8"))
self.addCleanup(self.kill, child)
(os.killpg if group else os.kill)(holder.pid, signum)
returned = holder.wait(timeout=30)
self.kill(child)
err = holder.stderr.read()
self.assertEqual(returned, code, err)
self.assertNotIn("Traceback", err)
self.assertTrue(self.lock_free())
def test_hold_runs_the_command_in_the_worktree_under_the_lock(self):
probe = (
"import fcntl, os, sys\n"
"with open(sys.argv[1], 'a', encoding='utf-8') as lock:\n"
" try:\n"
" fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB)\n"
" except BlockingIOError:\n"
" print('held', os.getcwd())\n"
"sys.exit(5)\n"
)
result = self.gate("hold", "--repo", str(self.repo), "--", sys.executable, "-c", probe, str(self.lock(RUN_LOCK)))
self.assertEqual((result.returncode, result.stdout), (5, f"held {self.repo}\n"), result.stderr)
self.assertTrue(self.lock_free())
result = self.gate("hold", "--repo", str(self.repo), "--", "sh", "-c", "kill -TERM $$")
self.assertEqual(result.returncode, 128 + signal.SIGTERM)
self.assertEqual(self.gate("hold", "--repo", str(self.repo)).returncode, 2)
def test_an_orphan_of_a_hold_keeps_the_lock(self):
pid_file = self.root / "deploy.pid"
holder = subprocess.Popen(
[sys.executable, str(SCRIPT), "hold", "--repo", str(self.repo), "--",
"sh", "-c", f"echo $$ > {pid_file}.tmp; mv {pid_file}.tmp {pid_file}; exec sleep 60"],
env=self.env, stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
)
self.addCleanup(holder.wait)
self.addCleanup(holder.kill)
deadline = time.monotonic() + 30
while not pid_file.exists():
self.assertIsNone(holder.poll())
self.assertLess(time.monotonic(), deadline)
time.sleep(0.05)
deploy = int(pid_file.read_text(encoding="utf-8"))
self.addCleanup(self.kill, deploy)
holder.kill()
holder.wait()
result = self.run_targets("test-a", extra=())
self.assertEqual(result.returncode, 75)
self.assertIn(f"another gate.py run or hold holds {self.repo} (pid {holder.pid}: "
f"gate.py hold --repo {self.repo} -- sh -c", result.stderr)
self.kill(deploy)
deadline = time.monotonic() + 30
while not self.lock_free():
self.assertLess(time.monotonic(), deadline, "the killed deploy kept the lock")
time.sleep(0.05)
self.assertEqual(self.run_targets("test-a", extra=()).returncode, 0)
def kill(self, pid: int) -> None:
with contextlib.suppress(ProcessLookupError):
os.kill(pid, signal.SIGKILL)
def test_a_run_waits_for_a_running_sync_instead_of_failing(self):
with (
open(self.lock(SYNC_LOCK), "a", encoding="utf-8") as sync,
open(self.lock(RUN_LOCK), "a", encoding="utf-8") as lock,
):
fcntl.flock(sync, fcntl.LOCK_EX)
sync.write(SYNC_HOLDER)
sync.flush()
fcntl.flock(lock, fcntl.LOCK_EX)
run = subprocess.Popen(
[sys.executable, str(SCRIPT), "run", "--repo", str(self.repo), "--target", "test-a",
"--log-dir", str(self.logs)],
env=self.env, stdin=subprocess.DEVNULL, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True,
)
self.addCleanup(run.wait)
self.addCleanup(run.kill)
self.assertEqual(
run.stderr.readline(),
f"gate: a hotfix.py sync merges in {self.repo} (pid 4242: hotfix.py sync --task t); waiting for it at "
f"most {load_gate().SYNC_WAIT_SECONDS} s\n",
)
out, err = run.communicate(timeout=60)
self.assertEqual(run.returncode, 0, err)
self.assertTrue(out.startswith("green\ttest-a\t"), out)
def test_a_sync_outlasting_the_wait_ends_it_with_75_naming_the_sync(self):
gate = load_gate()
gate.SYNC_WAIT_SECONDS = 1
with open(self.lock(SYNC_LOCK), "a", encoding="utf-8") as sync:
fcntl.flock(sync, fcntl.LOCK_EX)
sync.write(SYNC_HOLDER)
sync.flush()
with (
contextlib.redirect_stderr(io.StringIO()) as err,
self.assertRaises(SystemExit) as raised,
gate.worktree_lock(self.repo, ["gate.py", "hold"], []),
):
self.fail("the worktree lock was taken during a sync")
self.assertEqual(raised.exception.code, 75)
self.assertIn(f"a hotfix.py sync still holds {self.repo} after 1 s (pid 4242: hotfix.py sync --task t)",
err.getvalue())
self.assertTrue(self.lock_free())
class TestUsage(GateCase):
files = USAGE
tests/test_hotfix.py
+856
View File
@@ -0,0 +1,856 @@
"""Exercise kvb-git-hotfix: hotfix rows in the rasci register, pending, match, void and the funnel sync."""
from __future__ import annotations
import argparse
import contextlib
import fcntl
import importlib.util
import io
import json
import os
import shutil
import signal
import subprocess
import sys
import tempfile
import time
import types
import unittest
from pathlib import Path
from unittest import mock
REPO_ROOT = Path(__file__).resolve().parents[1]
HOTFIX = REPO_ROOT / "skills" / "kvb-git-hotfix" / "scripts" / "hotfix.py"
RASCI = REPO_ROOT / "skills" / "kvb-org-rasci" / "scripts" / "rasci.py"
GATE = REPO_ROOT / "skills" / "kvb-check-gate" / "scripts" / "gate.py"
PROBE = (
"import fcntl, sys\n"
"with open(sys.argv[1], 'a', encoding='utf-8') as lock:\n"
" try:\n"
" fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB)\n"
" except BlockingIOError:\n"
" print('held', sys.argv[2])\n"
" else:\n"
" print('free', sys.argv[2])\n"
)
SIGNATURE = r"connection refused .* port 5432"
RUN_LOCK = "kvb-gate-run.lock"
SYNC_LOCK = "kvb-gate-sync.lock"
IDENTITY = {
"GIT_AUTHOR_NAME": "Test",
"GIT_AUTHOR_EMAIL": "test@example.org",
"GIT_COMMITTER_NAME": "Test",
"GIT_COMMITTER_EMAIL": "test@example.org",
"GIT_CONFIG_NOSYSTEM": "1",
}
def load_hotfix() -> object:
spec = importlib.util.spec_from_file_location("hotfix", HOTFIX)
hotfix = importlib.util.module_from_spec(spec)
spec.loader.exec_module(hotfix)
return hotfix
class TestOriginUrls(unittest.TestCase):
def test_spellings_of_one_repository_normalize_alike(self):
hotfix = load_hotfix()
top = tempfile.mkdtemp()
self.addCleanup(shutil.rmtree, top)
for url in (
"https://github.com/Org/Repo.git",
"https://GitHub.com/org/REPO/",
"git@github.com:Org/Repo.git",
"ssh://git@github.com:22/Org/Repo.git/",
"https://user@github.com/Org/Repo",
):
with self.subTest(url=url):
self.assertEqual(hotfix.normalized(url, top), "github.com/org/repo")
self.assertEqual(hotfix.normalized("https://GitLab.com/Group/Sub/App.git", top), "gitlab.com/group/sub/app")
self.assertEqual(hotfix.normalized("https://git.example.org/Org/Repo.git", top), "git.example.org/Org/Repo")
for url in ("file:///srv/repo.git", "/srv/repo.git", "/srv/repo/"):
with self.subTest(url=url):
self.assertEqual(hotfix.normalized(url, top), os.path.realpath("/srv/repo"))
parent = os.path.dirname(os.path.realpath(top))
self.assertEqual(hotfix.normalized("../up.git", top), os.path.join(parent, "up"))
self.assertEqual(hotfix.normalized("up", top), os.path.join(os.path.realpath(top), "up"))
self.assertEqual(hotfix.normalized("", top), "")
self.assertNotEqual(hotfix.normalized("git@github.com:Org/Other.git", top), "github.com/org/repo")
def test_differing_origins_mean_different_repositories_even_in_one_git_dir(self):
hotfix = load_hotfix()
self.assertFalse(hotfix.same(("/x/.git", "github.com/a/x"), ("/x/.git", "github.com/b/y")))
self.assertTrue(hotfix.same(("/x/.git", "github.com/a/x"), ("/y/.git", "github.com/a/x")))
self.assertTrue(hotfix.same(("/x/.git", ""), ("/x/.git", "github.com/a/x")))
self.assertFalse(hotfix.same(("/x/.git", ""), ("/y/.git", "github.com/a/x")))
class TestSignatures(unittest.TestCase):
def test_only_runaway_repeats_are_refused(self):
hotfix = load_hotfix()
for pattern in ("(a+)+", "(?:x|y+)*z", "((ab)*c)+", r"(\d{2,})+"):
with self.subTest(pattern=pattern):
self.assertEqual(hotfix.risk(pattern), hotfix.NESTED)
for pattern in ("(a|a)*b", "(ab|a|b)*c", r"(\w|\d\d)+$", "([^x]|y)*z"):
with self.subTest(pattern=pattern):
self.assertEqual(hotfix.risk(pattern), hotfix.OVERLAP)
for pattern in (
r"(\d{1,3}\.){3}\d{1,3}", r"(\d{1,3}\.)+\d+", r"(ab)+ timeout \d+", "(foo|bar)+ failed",
r"(\d|x)+", "(foo|fob)*x", "(?>a+)*b", "(a++)*b", "port (5432|5433) refused",
):
with self.subTest(pattern=pattern):
self.assertIsNone(hotfix.risk(pattern))
class TestBoundedChildren(unittest.TestCase):
def setUp(self):
tmp = tempfile.TemporaryDirectory()
self.addCleanup(tmp.cleanup)
self.base = Path(tmp.name).resolve()
self.hotfix = load_hotfix()
def test_match_exits_2_when_the_search_worker_fails_never_no_match(self):
rows = {"hotfix/x": {"stage": "pr", "signature": "x", "branch": "fix/x", "sha": "0" * 40, "origin": "o"}}
register = types.SimpleNamespace(load=lambda: rows)
for name, patch in (
("no interpreter", mock.patch.object(self.hotfix.sys, "executable", str(self.base / "absent"))),
("worker exits 3", mock.patch.object(self.hotfix, "SEARCH", "import sys; sys.exit(3)")),
("worker answers short", mock.patch.object(self.hotfix, "SEARCH", "print('ready')")),
):
with self.subTest(name):
out, err = io.StringIO(), io.StringIO()
with (
patch,
contextlib.redirect_stdout(out),
contextlib.redirect_stderr(err),
self.assertRaises(SystemExit) as raised,
):
self.hotfix.cmd_match(register, argparse.Namespace(line="x", repo=None))
self.assertEqual(raised.exception.code, 2, err.getvalue())
self.assertEqual(out.getvalue(), "")
self.assertIn("signature search worker", err.getvalue())
def test_a_fetch_never_prompts_and_stops_after_fetch_seconds(self):
marker = self.base / "prompt"
wrapper = self.base / "bin" / "git"
wrapper.parent.mkdir()
wrapper.write_text(
f"#!/bin/sh\nprintf '%s' \"$GIT_TERMINAL_PROMPT\" > {marker}\nexec sleep 30\n", encoding="utf-8"
)
wrapper.chmod(0o755)
began = time.monotonic()
with (
mock.patch.object(self.hotfix, "FETCH_SECONDS", 1),
mock.patch.dict(os.environ, {"PATH": f"{wrapper.parent}:/usr/bin:/bin"}),
):
result = self.hotfix.fetch(str(self.base))
self.assertLess(time.monotonic() - began, 20)
self.assertEqual((result.returncode, result.stderr), (124, "stopped after 1 s"))
self.assertEqual(marker.read_text(encoding="utf-8"), "0")
class TestHotfix(unittest.TestCase):
def setUp(self):
tmp = tempfile.TemporaryDirectory()
self.addCleanup(tmp.cleanup)
self.base = Path(tmp.name).resolve()
home = self.base / "home"
sessions = home / ".claude" / "sessions"
sessions.mkdir(parents=True)
(sessions / "s1.json").write_text(json.dumps({"sessionId": "s1", "name": "One"}), encoding="utf-8")
self.root = self.base / "council"
self.env = {
"PATH": "/usr/bin:/bin",
"HOME": str(home),
"AGENTS_COUNCIL_ROOT": str(self.root),
"CLAUDE_CODE_SESSION_ID": "s1",
"GIT_CEILING_DIRECTORIES": str(self.base),
**IDENTITY,
}
self.origin = self.base / "origin.git"
self.seed = self.published(self.origin, self.base / "seed", "app.txt")
self.work = self.base / "work"
self.git(self.base, "clone", "-q", str(self.origin), str(self.work))
self.feature = self.base / "feature"
self.git(self.work, "worktree", "add", "-q", "-b", "feature", str(self.feature), "origin/main")
self.git(self.work, "switch", "-q", "-c", "line")
self.commit(self.work, "line.txt", "line\n", "line work")
self.git(self.work, "switch", "-q", "-c", "sidetree")
self.commit(self.work, "side.txt", "side\n", "side work")
self.git(self.work, "switch", "-q", "line")
self.commit(self.work, "line2.txt", "more\n", "more line work")
self.git(self.work, "switch", "-q", "sidetree")
self.rasci("set", "work/line", "--branch", "line", "--base", "main", "--workdir", str(self.work))
self.rasci("set", "work/side", "--branch", "sidetree", "--base", "line", "--workdir", str(self.work))
self.rasci("set", "work/feature", "--branch", "feature", "--workdir", str(self.feature))
self.git(self.seed, "switch", "-q", "-c", "fix/db")
self.fix = self.commit(self.seed, "db.txt", "retry\n", "fix db")
self.git(self.seed, "switch", "-q", "main")
self.git(self.seed, "merge", "-q", "--ff-only", "fix/db")
self.git(self.seed, "push", "-q", "origin", "main")
self.git(self.seed, "switch", "-q", "-c", "fix/lint")
self.lint = self.commit(self.seed, "lint.txt", "ok\n", "fix lint")
self.git(self.seed, "switch", "-q", "main")
self.out("add", "db", *self.flags({"--branch": "fix/db", "--sha": self.fix, "--signature": SIGNATURE,
"--status": "merged", "--target": "test-deploy", "--pr": "12"}))
self.out("add", "lint", *self.flags({"--branch": "fix/lint", "--sha": self.lint[:10],
"--signature": r"ruff: E\d+", "--target": "test-lint"}))
self.assertEqual(self.out("update", "lint", "--status", "pr", "--pr", "13"), ["updated hotfix/lint"])
self.other = self.published(self.base / "other.git", self.base / "other", "o.txt")
self.loose = self.git(self.other, "commit-tree", "HEAD^{tree}", "-p", "HEAD", "-m", "loose").strip()
self.git(self.other, "push", "-q", "origin", f"{self.loose}:refs/heads/main")
self.out("add", "other", *self.flags({"--repo": str(self.other), "--sha": self.loose,
"--signature": "disk full", "--status": "merged"}))
def published(self, origin: Path, repo: Path, name: str) -> Path:
self.git(self.base, "init", "-q", "--bare", "-b", "main", str(origin))
repo.mkdir()
self.git(repo, "init", "-q", "-b", "main")
self.commit(repo, name, "one\n", "init")
self.git(repo, "remote", "add", "origin", str(origin))
self.git(repo, "push", "-q", "-u", "origin", "main")
self.git(repo, "remote", "set-head", "origin", "main")
return repo
def git(self, cwd: Path, *args: str) -> str:
return subprocess.run(["git", *args], cwd=cwd, env=self.env, capture_output=True, text=True, check=True).stdout
def commit(self, repo: Path, name: str, text: str, message: str) -> str:
(repo / name).write_text(text, encoding="utf-8")
self.git(repo, "add", name)
self.git(repo, "commit", "-q", "-m", message)
return self.git(repo, "rev-parse", "HEAD").strip()
def flags(self, changes: dict[str, str | None]) -> list[str]:
values = {"--repo": str(self.seed), "--branch": "fix/x", "--sha": self.lint, "--signature": "x",
"--status": "staged", **changes}
return [part for flag, value in values.items() if value is not None for part in (flag, value)]
def hotfix(self, *args: str, env: dict[str, str] | None = None) -> subprocess.CompletedProcess:
return subprocess.run(
[sys.executable, str(HOTFIX), *args],
capture_output=True, text=True, env=env or self.env, cwd=self.base, check=False,
)
def out(self, *args: str) -> list[str]:
result = self.hotfix(*args)
self.assertEqual(result.returncode, 0, result.stderr)
return result.stdout.splitlines()
def rasci(self, *args: str) -> list[str]:
result = subprocess.run(
[sys.executable, str(RASCI), *args],
capture_output=True, text=True, env=self.env, cwd=self.base, check=False,
)
self.assertEqual(result.returncode, 0, result.stderr)
return result.stdout.splitlines()
def rows(self) -> dict[str, dict]:
return json.loads((self.root / "rasci.json").read_text(encoding="utf-8"))
def write_rows(self, rows: dict[str, dict]) -> None:
(self.root / "rasci.json").write_text(json.dumps(rows), encoding="utf-8")
def sync(self, repo: Path, task: str, *extra: str) -> subprocess.CompletedProcess:
return self.hotfix("sync", "--repo", str(repo), "--task", task, *extra)
def merges(self, result: subprocess.CompletedProcess) -> list[list[str]]:
self.assertEqual(result.returncode, 0, result.stderr)
return [line.split("\t")[:2] for line in result.stdout.splitlines()]
def head(self, repo: Path) -> str:
return self.git(repo, "rev-parse", "HEAD").strip()
def contains(self, repo: Path, sha: str) -> bool:
return subprocess.run(
["git", "merge-base", "--is-ancestor", sha, "HEAD"], cwd=repo, env=self.env, capture_output=True,
check=False,
).returncode == 0
def test_rows_are_register_tasks_with_the_author_as_r_and_a(self):
rows = self.rows()
db = rows["hotfix/db"]
keys = ("common_dir", "origin", "branch", "sha", "pr", "signature", "targets", "stage", "status", "roles")
self.assertEqual(
{key: db[key] for key in keys},
{"common_dir": str(self.seed / ".git"), "origin": str(self.base / "origin"), "branch": "fix/db",
"sha": self.fix, "pr": 12, "signature": SIGNATURE, "targets": ["test-deploy"], "stage": "merged",
"status": "done", "roles": {"One": "RA"}},
)
self.assertNotIn("repo", db)
lint = rows["hotfix/lint"]
self.assertEqual((lint["sha"], lint["stage"], lint["status"], lint["pr"]), (self.lint, "pr", "running", 13))
self.assertEqual(rows["hotfix/other"]["targets"], [])
shown = "\n".join(self.rasci("show"))
self.assertIn("| hotfix/lint | running | - | One |", shown)
self.assertNotIn("hotfix/db", shown)
def test_pending_reports_missing_and_open_hotfixes_of_the_repository(self):
expected = [
f"missing (merge main)\thotfix/db\tmerged\tfix/db\t{self.fix}\t#12",
f"open, not merged\thotfix/lint\tpr\tfix/lint\t{self.lint}\t#13",
"hotfix: 1 missing, 1 open",
]
for repo in (self.work, self.feature):
with self.subTest(repo=repo.name):
result = self.hotfix("pending", "--repo", str(repo))
self.assertEqual((result.returncode, result.stdout.splitlines()), (1, expected), result.stderr)
self.assertEqual(self.out("pending", "--repo", str(self.seed)), [expected[1], "hotfix: 0 missing, 1 open"])
self.assertEqual(self.sync(self.work, "work/side").returncode, 0)
self.assertEqual(self.out("pending", "--repo", str(self.work)), [expected[1], "hotfix: 0 missing, 1 open"])
copy = self.base / "copy"
self.git(self.base, "clone", "-q", f"file://{self.origin}", str(copy))
self.assertEqual(self.out("pending", "--repo", str(copy)), [expected[1], "hotfix: 0 missing, 1 open"])
def test_a_row_outlives_the_worktree_it_was_recorded_in(self):
self.git(self.work, "fetch", "-q", "origin")
tree = self.base / "fixtree"
self.git(self.work, "worktree", "add", "-q", "-b", "fix/tree", str(tree), "origin/main")
sha = self.commit(tree, "tree.txt", "tree\n", "fix tree")
self.out("add", "tree", *self.flags({"--repo": str(tree), "--branch": "fix/tree", "--sha": sha,
"--target": "test-tree"}))
self.git(tree, "push", "-q", "origin", "HEAD:main")
self.git(self.work, "worktree", "remove", "--force", str(tree))
self.assertEqual(self.out("update", "tree", "--status", "merged", "--sha", sha), ["updated hotfix/tree"])
line = f"missing (merge main)\thotfix/tree\tmerged\tfix/tree\t{sha}\t-"
copy = self.base / "copy"
self.git(self.base, "clone", "-q", f"file://{self.origin}", str(copy))
self.git(copy, "reset", "-q", "--hard", "HEAD~1")
for repo in (self.work, self.feature, self.seed, copy):
with self.subTest(repo=repo.name):
result = self.hotfix("pending", "--repo", str(repo))
self.assertEqual((result.returncode, result.stderr), (1, ""))
self.assertIn(line, result.stdout.splitlines())
result = self.hotfix("void", "--repo", str(copy), "--target", "test-tree")
self.assertEqual((result.returncode, result.stdout), (3, f"void\thotfix/tree\t{sha}\ttest-tree\n"))
def test_a_row_of_an_unknown_repository_is_named_on_stderr_and_changes_no_exit_code(self):
rows = self.rows()
legacy = {key: value for key, value in rows["hotfix/db"].items() if key not in ("common_dir", "origin")}
rows["hotfix/legacy"] = {**legacy, "repo": str(self.base / "gone"), "targets": ["test-legacy"]}
self.write_rows(rows)
hint = (
"hotfix/legacy: repository unknown (no origin URL, git dir - gone); record it with "
f"`hotfix.py update legacy --status merged --sha {self.fix} --repo <clone>`"
)
result = self.hotfix("pending", "--repo", str(self.work))
self.assertEqual(result.returncode, 1)
self.assertIn(hint, result.stderr)
self.assertIn(f"missing (merge main)\thotfix/db\tmerged\tfix/db\t{self.fix}\t#12", result.stdout)
result = self.hotfix("void", "--repo", str(self.work), "--target", "test-lint")
self.assertEqual((result.returncode, result.stdout), (0, ""))
self.assertIn(hint, result.stderr)
self.assertEqual(self.hotfix("void", "--repo", str(self.work), "--target", "test-deploy").returncode, 3)
self.out("update", "legacy", "--status", "merged", "--sha", self.fix, "--repo", str(self.work))
result = self.hotfix("pending", "--repo", str(self.work))
self.assertEqual((result.returncode, result.stderr), (1, ""))
self.assertIn(f"missing (merge main)\thotfix/legacy\tmerged\tfix/db\t{self.fix}\t#12", result.stdout)
def test_update_resolves_the_sha_in_the_recorded_git_dir_or_an_explicit_clone(self):
lone = self.base / "lone"
self.git(self.base, "clone", "-q", str(self.origin), str(lone))
sha = self.commit(lone, "lone.txt", "lone\n", "fix lone")
self.out("add", "lone", *self.flags({"--repo": str(lone), "--branch": "fix/lone", "--sha": sha}))
self.git(lone, "push", "-q", "origin", "HEAD:main")
shutil.rmtree(lone)
self.assertEqual(self.out("update", "lone", "--status", "pr", "--pr", "7"), ["updated hotfix/lone"])
result = self.hotfix("update", "lone", "--status", "merged", "--sha", sha)
self.assertEqual(result.returncode, 2)
self.assertIn(f"hotfix/lone: its git dir {lone / '.git'} is gone; pass --repo", result.stderr)
result = self.hotfix("update", "lone", "--status", "merged", "--sha", sha, "--repo", str(self.other))
self.assertEqual(result.returncode, 2)
self.assertIn("is another repository than hotfix/lone's", result.stderr)
self.out("update", "lone", "--status", "merged", "--sha", sha, "--repo", str(self.seed))
row = self.rows()["hotfix/lone"]
self.assertEqual((row["common_dir"], row["stage"], row["sha"], row["pr"]), (str(self.seed / ".git"), "merged", sha, 7))
def squash_merged(self) -> tuple[str, str]:
self.git(self.seed, "switch", "-q", "-c", "fix/sq")
branch_sha = self.commit(self.seed, "sq.txt", "sq\n", "fix sq")
self.git(self.seed, "switch", "-q", "main")
self.out("add", "sq", *self.flags({"--branch": "fix/sq", "--sha": branch_sha, "--status": "pr",
"--target": "test-sq"}))
self.git(self.seed, "merge", "-q", "--squash", "fix/sq")
self.git(self.seed, "commit", "-q", "-m", "fix sq, squashed")
self.git(self.seed, "push", "-q", "origin", "main")
return branch_sha, self.head(self.seed)
def test_merged_needs_the_sha_as_it_sits_on_the_default_branch(self):
branch_sha, squashed = self.squash_merged()
result = self.hotfix("update", "sq", "--status", "merged")
self.assertEqual(result.returncode, 2)
self.assertIn("--status merged needs --sha", result.stderr)
for args in (
("update", "sq", "--status", "merged", "--sha", branch_sha),
("add", "sq2", *self.flags({"--branch": "fix/sq", "--sha": branch_sha, "--status": "merged"})),
):
with self.subTest(args=args[0]):
result = self.hotfix(*args)
self.assertEqual(result.returncode, 2)
self.assertIn(f"--sha {branch_sha} is not on origin/main", result.stderr)
self.assertEqual(self.rows()["hotfix/sq"]["stage"], "pr")
self.out("update", "sq", "--status", "merged", "--sha", squashed)
self.assertEqual(self.rows()["hotfix/sq"]["sha"], squashed)
def test_a_merged_sha_off_the_default_branch_warns_and_never_voids(self):
branch_sha, squashed = self.squash_merged()
self.out("update", "sq", "--status", "merged", "--sha", squashed)
rows = self.rows()
rows["hotfix/sq"]["sha"] = branch_sha
self.write_rows(rows)
for _ in range(2):
self.assertEqual(self.sync(self.work, "work/side").returncode, 0)
result = self.hotfix("void", "--repo", str(self.work), "--target", "test-sq")
self.assertEqual((result.returncode, result.stdout), (0, ""))
self.assertIn(f"hotfix/sq: {branch_sha} is not on origin/main", result.stderr)
result = self.hotfix("pending", "--repo", str(self.work))
self.assertEqual(result.returncode, 0, result.stderr)
self.assertIn(f"not on origin/main\thotfix/sq\tmerged\tfix/sq\t{branch_sha}\t-", result.stdout.splitlines())
def test_match_finds_hotfixes_by_their_signature(self):
origin = str(self.base / "origin")
self.assertEqual(
self.out("match", "--line", "psql: connection refused to db at port 5432"),
[f"hotfix/db\tmerged\tfix/db\t{self.fix}\t#12\t{origin}"],
)
self.assertEqual(
self.out("match", "--line", "ruff: E501 line too long", "--repo", str(self.work)),
[f"hotfix/lint\tpr\tfix/lint\t{self.lint}\t#13\t{origin}"],
)
result = self.hotfix("match", "--line", "all green")
self.assertEqual((result.returncode, result.stdout), (1, "hotfix: no signature matches\n"))
foreign = f"hotfix/other\tmerged\tfix/x\t{self.loose}\t-\t{self.base / 'other'}"
result = self.hotfix("match", "--line", "disk full on /var", "--repo", str(self.work))
self.assertEqual(
(result.returncode, result.stdout.splitlines()),
(1, [f"foreign\t{foreign}", "hotfix: no signature of this repository matches"]),
)
self.assertEqual(self.out("match", "--line", "disk full on /var", "--repo", str(self.other)), [foreign])
def test_add_refuses_only_a_signature_that_backtracks_without_end(self):
for pattern, reason in (("(a+)+", "repeats a group that repeats itself"),
("(a|a)*b", "repeats an alternation whose branches can start alike")):
with self.subTest(pattern=pattern):
result = self.hotfix("add", "nest", *self.flags({"--signature": pattern}))
self.assertEqual(result.returncode, 2)
self.assertIn(reason, result.stderr)
self.out("add", "plain", *self.flags({"--signature": r"(ab)+ timeout \d+"}))
self.out("add", "address", *self.flags({"--signature": r"no route to (\d{1,3}\.){3}\d{1,3}"}))
def test_match_names_a_signature_that_runs_too_long_and_searches_the_rest(self):
rows = self.rows()
rows["hotfix/slow"] = {**rows["hotfix/lint"], "signature": "(x+x+)+y"}
rows["hotfix/tail"] = {**rows["hotfix/lint"], "signature": "x{40}"}
self.write_rows(rows)
began = time.monotonic()
result = self.hotfix("match", "--line", "ruff: E501 " + "x" * 40, "--repo", str(self.work))
self.assertLess(time.monotonic() - began, 30)
self.assertEqual(result.returncode, 0, result.stderr)
self.assertEqual(result.stdout.splitlines(), [f"{task}\tpr\tfix/lint\t{self.lint}\t#13\t{self.base / 'origin'}"
for task in ("hotfix/lint", "hotfix/tail")])
self.assertIn("hotfix/slow: signature '(x+x+)+y' still searched this line at the 3 s deadline, skipped",
result.stderr)
def test_void_only_for_a_missing_merged_hotfix_of_the_targets(self):
result = self.hotfix("void", "--repo", str(self.work), "--target", "test", "--target", "test-deploy")
self.assertEqual((result.returncode, result.stdout), (3, f"void\thotfix/db\t{self.fix}\ttest-deploy\n"))
result = self.hotfix("void", "--repo", str(self.work), "--target", "test-lint")
self.assertEqual((result.returncode, result.stdout), (0, ""))
self.assertEqual(self.sync(self.work, "work/side").returncode, 0)
self.assertEqual(self.hotfix("void", "--repo", str(self.work), "--target", "test-deploy").returncode, 0)
self.git(self.seed, "switch", "-q", "-c", "fix/any")
anything = self.commit(self.seed, "any.txt", "any\n", "fix any")
self.git(self.seed, "push", "-q", "origin", "HEAD:main")
self.out("add", "any", *self.flags({"--branch": "fix/any", "--sha": anything, "--status": "merged"}))
for target in ("test-lint", "deploy"):
with self.subTest(target=target):
result = self.hotfix("void", "--repo", str(self.work), "--target", target)
self.assertEqual((result.returncode, result.stdout), (3, f"void\thotfix/any\t{anything}\tall targets\n"))
def test_test_star_targets_void_the_umbrella_and_a_deploy_needs_deploy(self):
result = self.hotfix("void", "--repo", str(self.work), "--target", "test")
self.assertEqual((result.returncode, result.stdout), (3, f"void\thotfix/db\t{self.fix}\ttest-deploy\n"))
self.assertEqual(self.hotfix("void", "--repo", str(self.work), "--target", "deploy").returncode, 0)
self.git(self.seed, "switch", "-q", "-c", "fix/deploy")
rollout = self.commit(self.seed, "deploy.txt", "deploy\n", "fix deploy")
self.git(self.seed, "push", "-q", "origin", "HEAD:main")
self.out("add", "deploy", *self.flags({"--branch": "fix/deploy", "--sha": rollout, "--status": "merged",
"--target": "deploy"}))
result = self.hotfix("void", "--repo", str(self.work), "--target", "deploy")
self.assertEqual((result.returncode, result.stdout), (3, f"void\thotfix/deploy\t{rollout}\tdeploy\n"))
result = self.hotfix("void", "--repo", str(self.work), "--target", "test-lint")
self.assertEqual((result.returncode, result.stdout), (0, ""))
def test_sync_merges_main_then_the_base_chain_and_skips_contained_refs(self):
hooks = self.work / ".git" / "hooks"
for hook in ("pre-merge-commit", "commit-msg"):
(hooks / hook).write_text("#!/bin/sh\nexit 1\n", encoding="utf-8")
(hooks / hook).chmod(0o755)
result = self.sync(self.work, "work/side")
self.assertEqual(self.merges(result), [["merged", "origin/main"], ["merged", "line"]])
self.assertEqual(result.stdout.splitlines()[1].split("\t")[2], self.head(self.work))
for name in ("db.txt", "line2.txt", "side.txt"):
self.assertTrue((self.work / name).is_file(), name)
self.assertEqual(self.git(self.work, "rev-list", "--count", "--merges", "HEAD").strip(), "2")
self.assertEqual(self.merges(self.sync(self.work, "work/side")), [["contained", "origin/main"],
["contained", "line"]])
result = self.sync(self.feature, "work/feature")
self.assertEqual((result.returncode, result.stdout), (0, f"merged\torigin/main\t{self.fix}\n"), result.stderr)
def test_the_base_chain_follows_only_open_rows_of_this_repository(self):
self.git(self.other, "branch", "line")
gone = self.base / "gone"
self.rasci("set", "work/gui", "--branch", "line", "--base", "feature", "--workdir", str(self.other))
self.rasci("set", "work/old", "--branch", "line", "--base", "fix/db", "--workdir", str(self.work),
"--status", "done")
self.rasci("set", "work/ghost", "--branch", "line", "--base", "feature", "--workdir", str(gone))
result = self.sync(self.work, "work/side")
self.assertEqual(self.merges(result), [["merged", "origin/main"], ["merged", "line"]])
self.assertIn(f"work/ghost: workdir {gone} is no git repository, its base is not followed", result.stderr)
def test_a_chain_row_without_workdir_counts_when_its_branch_resolves_here(self):
self.rasci("set", "work/line", "--workdir", "", "--base", "feature")
self.assertEqual(self.merges(self.sync(self.work, "work/side")),
[["merged", "origin/main"], ["merged", "line"], ["contained", "feature"]])
self.rasci("set", "work/up", "--branch", "nowhere", "--base", "main")
self.rasci("set", "work/side", "--base", "nowhere")
before = self.head(self.work)
result = self.sync(self.work, "work/side")
self.assertEqual((result.returncode, result.stdout), (2, ""))
self.assertIn("work/up records no workdir and its branch nowhere resolves neither as nowhere nor as "
f"origin/nowhere in {self.work}; record it with `rasci.py set work/up --workdir <dir>`",
result.stderr)
self.assertEqual(self.head(self.work), before)
def test_sync_without_an_origin_merges_local_branches_and_skips_the_fetch(self):
lone = self.base / "lone"
lone.mkdir()
self.git(lone, "init", "-q", "-b", "main")
self.commit(lone, "a.txt", "a\n", "init")
self.git(lone, "switch", "-q", "-c", "line")
self.commit(lone, "l1.txt", "l1\n", "line one")
self.git(lone, "switch", "-q", "-c", "feat")
self.commit(lone, "f.txt", "f\n", "feat")
self.git(lone, "switch", "-q", "line")
self.commit(lone, "l2.txt", "l2\n", "line two")
self.git(lone, "switch", "-q", "main")
self.commit(lone, "m.txt", "m\n", "main two")
self.git(lone, "switch", "-q", "feat")
self.rasci("set", "work/lone", "--branch", "feat", "--base", "line", "--workdir", str(lone))
self.rasci("set", "work/loneline", "--branch", "line", "--base", "main", "--workdir", str(lone))
result = self.sync(lone, "work/lone")
self.assertEqual(result.returncode, 2)
self.assertIn(f"{lone} has no origin remote; pass --default <branch>", result.stderr)
self.assertEqual(self.merges(self.sync(lone, "work/lone", "--default", "main")),
[["merged", "main"], ["merged", "line"]])
for name in ("m.txt", "l2.txt", "f.txt"):
self.assertTrue((lone / name).is_file(), name)
def test_relative_origins_resolve_against_each_worktree(self):
clones = []
for name in ("p1", "p2"):
clone = self.base / name / "app"
clone.mkdir(parents=True)
self.git(clone, "init", "-q", "-b", "main")
self.git(clone, "remote", "add", "origin", "../up")
clones.append((clone, self.commit(clone, "a.txt", f"{name}\n", "init")))
(one, sha), (two, _) = clones
self.out("add", "rel", *self.flags({"--repo": str(one), "--branch": "fix/rel", "--sha": sha,
"--signature": "relative boom"}))
self.assertEqual(self.rows()["hotfix/rel"]["origin"], str(self.base / "p1" / "up"))
found = f"hotfix/rel\tstaged\tfix/rel\t{sha}\t-\t{self.base / 'p1' / 'up'}"
self.assertEqual(self.out("match", "--line", "relative boom", "--repo", str(one)), [found])
result = self.hotfix("match", "--line", "relative boom", "--repo", str(two))
self.assertEqual((result.returncode, result.stdout.splitlines()[0]), (1, f"foreign\t{found}"))
def test_another_origin_in_the_same_git_dir_is_another_repository(self):
lint = f"hotfix/lint\tpr\tfix/lint\t{self.lint}\t#13\t{self.base / 'origin'}"
self.assertEqual(self.out("match", "--line", "ruff: E501", "--repo", str(self.seed)), [lint])
self.git(self.seed, "remote", "set-url", "origin", str(self.base / "elsewhere.git"))
result = self.hotfix("match", "--line", "ruff: E501", "--repo", str(self.seed))
self.assertEqual((result.returncode, result.stdout.splitlines()[0]), (1, f"foreign\t{lint}"))
def test_origins_apply_insteadof_and_fold_case_on_caseless_hosts(self):
first = self.base / "gh1"
first.mkdir()
self.git(first, "init", "-q", "-b", "main")
self.git(first, "config", "url.https://github.com/.insteadOf", "gh:")
self.git(first, "remote", "add", "origin", "gh:Org/App.git")
sha = self.commit(first, "a.txt", "a\n", "init")
self.out("add", "gh", *self.flags({"--repo": str(first), "--branch": "fix/gh", "--sha": sha,
"--signature": "github boom"}))
self.assertEqual(self.rows()["hotfix/gh"]["origin"], "github.com/org/app")
second = self.base / "gh2"
self.git(self.base, "clone", "-q", str(first), str(second))
self.git(second, "remote", "set-url", "origin", "https://GitHub.com/ORG/app/")
self.assertEqual(self.out("match", "--line", "github boom", "--repo", str(second)),
[f"hotfix/gh\tstaged\tfix/gh\t{sha}\t-\tgithub.com/org/app"])
def test_pending_filters_by_targets_as_void_does(self):
lint = f"open, not merged\thotfix/lint\tpr\tfix/lint\t{self.lint}\t#13"
result = self.hotfix("pending", "--repo", str(self.work), "--target", "test-lint")
self.assertEqual((result.returncode, result.stdout.splitlines()), (0, [lint, "hotfix: 0 missing, 1 open"]))
result = self.hotfix("pending", "--repo", str(self.work), "--target", "deploy")
self.assertEqual((result.returncode, result.stdout.splitlines()), (0, ["hotfix: 0 missing, 0 open"]))
result = self.hotfix("pending", "--repo", str(self.work), "--target", "test")
self.assertEqual(result.returncode, 1, result.stderr)
self.assertIn(f"missing (merge main)\thotfix/db\tmerged\tfix/db\t{self.fix}\t#12", result.stdout)
def test_a_base_merges_from_whichever_of_origin_and_local_contains_the_other(self):
self.git(self.work, "push", "-q", "origin", "line")
upstream = self.git(self.work, "commit-tree", "line^{tree}", "-p", "line", "-m", "upstream").strip()
self.git(self.work, "push", "-q", "origin", f"{upstream}:refs/heads/line")
self.assertEqual(self.merges(self.sync(self.work, "work/side")),
[["merged", "origin/main"], ["merged", "origin/line"]])
self.assertTrue(self.contains(self.work, upstream))
ahead = self.git(self.work, "commit-tree", f"{upstream}^{{tree}}", "-p", upstream, "-m", "ahead").strip()
self.git(self.work, "update-ref", "refs/heads/line", ahead)
self.assertEqual(self.merges(self.sync(self.work, "work/side")),
[["contained", "origin/main"], ["merged", "line"]])
apart = self.git(self.work, "commit-tree", f"{upstream}^{{tree}}", "-p", upstream, "-m", "apart").strip()
self.git(self.work, "push", "-q", "origin", f"{apart}:refs/heads/line")
before = self.head(self.work)
result = self.sync(self.work, "work/side")
self.assertEqual((result.returncode, result.stdout), (2, ""))
self.assertIn("line and origin/line have diverged", result.stderr)
self.assertEqual(self.head(self.work), before)
self.git(self.work, "branch", "-D", "line")
self.assertEqual(self.merges(self.sync(self.work, "work/side")),
[["contained", "origin/main"], ["merged", "origin/line"]])
self.assertTrue(self.contains(self.work, apart))
def test_sync_aborts_the_first_conflicting_merge_with_exit_4(self):
line = self.base / "line"
self.git(self.work, "worktree", "add", "-q", str(line), "line")
self.commit(line, "side.txt", "other\n", "clashing line work")
result = self.sync(self.work, "work/side")
self.assertEqual(result.returncode, 4, result.stderr)
self.assertEqual([row.split("\t")[0] for row in result.stdout.splitlines()], ["merged", "conflict"])
self.assertEqual(result.stdout.splitlines()[1], "conflict\tline\tside.txt")
self.assertIn("merging line conflicts in side.txt; merge aborted", result.stderr)
self.assertFalse((self.work / ".git" / "MERGE_HEAD").exists())
self.assertEqual(self.git(self.work, "status", "--porcelain", "--untracked-files=no"), "")
self.assertTrue(self.contains(self.work, self.fix))
self.assertFalse(self.contains(self.work, self.head(line)))
def test_a_merge_failing_without_a_conflict_exits_2_with_gits_message(self):
before = self.head(self.work)
(self.work / "db.txt").write_text("untracked\n", encoding="utf-8")
result = self.sync(self.work, "work/side")
self.assertEqual((result.returncode, result.stdout), (2, ""))
self.assertIn("untracked working tree files would be overwritten by merge", result.stderr)
(self.work / "db.txt").unlink()
self.git(self.work, "config", "commit.gpgsign", "true")
self.git(self.work, "config", "gpg.program", "false")
result = self.sync(self.work, "work/side")
self.assertEqual((result.returncode, result.stdout), (2, ""))
self.assertIn("gpg failed to sign the data", result.stderr)
self.assertFalse((self.work / ".git" / "MERGE_HEAD").exists())
self.assertEqual(self.git(self.work, "status", "--porcelain", "--untracked-files=no"), "")
self.assertEqual(self.head(self.work), before)
def test_sync_needs_the_tasks_branch_checked_out(self):
before = self.head(self.work)
self.git(self.work, "switch", "-q", "--detach")
result = self.sync(self.work, "work/side")
self.assertEqual(result.returncode, 2)
self.assertIn(f"{self.work} has a detached HEAD checked out, work/side works on sidetree", result.stderr)
self.git(self.work, "switch", "-q", "line")
line = self.head(self.work)
result = self.sync(self.work, "work/side")
self.assertEqual(result.returncode, 2)
self.assertIn(f"{self.work} has line checked out, work/side works on sidetree", result.stderr)
self.assertEqual((self.git(self.work, "rev-parse", "sidetree").strip(), self.head(self.work)), (before, line))
self.rasci("set", "work/loose")
self.assertIn("work/loose records no branch", self.sync(self.work, "work/loose").stderr)
def test_sync_refuses_while_a_gate_or_a_busy_process_holds_the_worktree(self):
before = self.head(self.work)
git_dir = Path(self.git(self.work, "rev-parse", "--absolute-git-dir").strip())
with open(git_dir / RUN_LOCK, "a", encoding="utf-8") as lock:
fcntl.flock(lock, fcntl.LOCK_EX)
result = self.sync(self.work, "work/side")
self.assertEqual(result.returncode, 2)
self.assertIn(f"a gate.py run or hold holds {self.work}", result.stderr)
config = self.base / "config.json"
config.write_text(json.dumps({"busy_processes": ["sleep"]}), encoding="utf-8")
self.env["AGENTS_COUNCIL_CONFIG"] = str(config)
deploy = subprocess.Popen(["sleep", "60"], cwd=self.work)
self.addCleanup(deploy.wait)
self.addCleanup(deploy.kill)
result = self.sync(self.work, "work/side")
self.assertEqual(result.returncode, 2)
self.assertIn(f"a busy process (watch.py busy_processes) runs in {self.work}", result.stderr)
self.assertEqual(self.head(self.work), before)
deploy.kill()
deploy.wait()
self.assertEqual(self.sync(self.work, "work/side").returncode, 0)
def wait_for(self, path: Path, process: subprocess.Popen) -> None:
deadline = time.monotonic() + 30
while not path.exists():
if process.poll() is not None:
self.fail(f"exited {process.returncode} before {path} appeared: {process.stderr.read()}")
self.assertLess(time.monotonic(), deadline, f"{path} never appeared")
time.sleep(0.05)
def kill(self, pid: int) -> None:
with contextlib.suppress(ProcessLookupError):
os.kill(pid, signal.SIGKILL)
def lock_free(self, lock: Path) -> bool:
with open(lock, "a", encoding="utf-8") as handle:
try:
fcntl.flock(handle, fcntl.LOCK_EX | fcntl.LOCK_NB)
except BlockingIOError:
return False
return True
def test_sync_refuses_while_an_orphan_of_gate_py_hold_runs_where_no_process_check_sees_it(self):
before = self.head(self.work)
pid_file, cwd_file = self.base / "deploy.pid", self.base / "deploy.cwd"
holder = subprocess.Popen(
[sys.executable, str(GATE), "hold", "--repo", str(self.work), "--",
"sh", "-c", f"pwd > {cwd_file}; echo $$ > {pid_file}.tmp; mv {pid_file}.tmp {pid_file}; exec sleep 60"],
env=self.env, stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL, stderr=subprocess.PIPE, text=True,
)
self.addCleanup(holder.stderr.close)
self.addCleanup(holder.wait)
self.addCleanup(holder.kill)
self.wait_for(pid_file, holder)
deploy = int(pid_file.read_text(encoding="utf-8"))
self.addCleanup(self.kill, deploy)
self.assertEqual(cwd_file.read_text(encoding="utf-8").strip(), str(self.work))
holder.kill()
holder.wait()
result = self.sync(self.work, "work/side")
self.assertEqual(result.returncode, 2, result.stdout)
self.assertIn(f"a gate.py run or hold holds {self.work} (pid {holder.pid}: "
f"gate.py hold --repo {self.work} -- sh", result.stderr)
self.assertEqual(self.head(self.work), before)
lock = Path(self.git(self.work, "rev-parse", "--absolute-git-dir").strip()) / RUN_LOCK
os.kill(deploy, signal.SIGKILL)
deadline = time.monotonic() + 30
while not self.lock_free(lock):
self.assertLess(time.monotonic(), deadline, "the killed deploy kept the lock")
time.sleep(0.05)
self.assertEqual(self.sync(self.work, "work/side").returncode, 0)
def test_sync_fetches_before_the_locks_and_holds_them_through_every_merge_and_abort(self):
line = self.base / "line"
self.git(self.work, "worktree", "add", "-q", str(line), "line")
self.commit(line, "side.txt", "other\n", "clashing line work")
lock = Path(self.git(self.work, "rev-parse", "--absolute-git-dir").strip()) / RUN_LOCK
probe, calls = self.base / "probe.py", self.base / "lock-states.log"
probe.write_text(PROBE, encoding="utf-8")
wrapper = self.base / "bin" / "git"
wrapper.parent.mkdir()
real = shutil.which("git", path=self.env["PATH"])
wrapper.write_text(
"#!/bin/sh\n"
f"case \" $* \" in *\" fetch \"*|*\" merge \"*) {sys.executable} {probe} {lock} \"$*\" >> {calls} ;; esac\n"
f"exec {real} \"$@\"\n",
encoding="utf-8",
)
wrapper.chmod(0o755)
traced = {**self.env, "PATH": f"{wrapper.parent}:{self.env['PATH']}"}
result = self.hotfix("sync", "--repo", str(self.work), "--task", "work/side", env=traced)
self.assertEqual(result.returncode, 4, result.stderr)
states = [entry.split(" ", 1) for entry in calls.read_text(encoding="utf-8").splitlines()]
commands = [command.split(" ")[2] + (" --abort" if command.endswith("--abort") else "") for _, command in states]
self.assertEqual(commands, ["fetch", "merge", "merge", "merge --abort"])
self.assertEqual([state for state, _ in states], ["free", "held", "held", "held"], states)
self.assertTrue(self.lock_free(lock))
self.assertIn(f"\thotfix.py sync --repo {self.work} --task work/side\n",
lock.with_name(SYNC_LOCK).read_text(encoding="utf-8"))
def test_sync_and_update_keep_git_maintenance_off(self):
calls = self.base / "git-calls.log"
wrapper = self.base / "bin" / "git"
wrapper.parent.mkdir()
real = shutil.which("git", path=self.env["PATH"])
wrapper.write_text(f"#!/bin/sh\nprintf '%s\\n' \"$*\" >> {calls}\nexec {real} \"$@\"\n", encoding="utf-8")
wrapper.chmod(0o755)
traced = {**self.env, "PATH": f"{wrapper.parent}:{self.env['PATH']}"}
self.assertEqual(self.hotfix("sync", "--repo", str(self.work), "--task", "work/side", env=traced).returncode, 0)
result = self.hotfix("update", "lint", "--status", "merged", "--sha", self.fix, env=traced)
self.assertEqual(result.returncode, 0, result.stderr)
lines = calls.read_text(encoding="utf-8").splitlines()
self.assertTrue(any(line.startswith("-c maintenance.auto=false merge ") for line in lines), lines)
self.assertEqual([line for line in lines if not line.startswith("-c maintenance.auto=false ")], [])
fetches = [line for line in lines if " fetch " in f" {line} "]
self.assertEqual(len(fetches), 2, lines)
self.assertTrue(all("--no-auto-maintenance" in line for line in fetches), fetches)
def test_sync_needs_a_clean_tree_and_names_the_default_branch_explicitly(self):
before = self.head(self.work)
(self.work / "side.txt").write_text("changed\n", encoding="utf-8")
result = self.sync(self.work, "work/side")
self.assertEqual(result.returncode, 2)
self.assertIn("tracked files are dirty, commit them first: side.txt", result.stderr)
self.assertEqual(self.head(self.work), before)
(self.work / "side.txt").write_text("side\n", encoding="utf-8")
(self.work / "notes.txt").write_text("untracked\n", encoding="utf-8")
self.git(self.work, "remote", "set-head", "origin", "-d")
result = self.sync(self.work, "work/side")
self.assertEqual(result.returncode, 2)
self.assertIn("pass --default <branch>", result.stderr)
self.assertEqual(self.head(self.work), before)
self.assertEqual(self.sync(self.work, "work/side", "--default", "main").returncode, 0)
self.git(self.work, "remote", "set-head", "origin", "main")
result = self.sync(self.work, "work/side", "--default", "line")
self.assertEqual(result.returncode, 2)
self.assertIn("origin/HEAD names main, --default says line", result.stderr)
def test_invalid_input_exits_2_and_leaves_the_register_alone(self):
register = self.root / "rasci.json"
before = register.read_text(encoding="utf-8")
for args in (
("add", "Bad Slug", *self.flags({})),
("add", "db", *self.flags({})),
("add", "x", *self.flags({"--repo": "seed"})),
("add", "x", *self.flags({"--repo": str(self.base / "home")})),
("add", "x", *self.flags({"--sha": "HEAD"})),
("add", "x", *self.flags({"--sha": "0" * 40})),
("add", "x", *self.flags({"--signature": "("})),
("add", "x", *self.flags({"--signature": " "})),
("add", "x", *self.flags({"--signature": "(a+)+"})),
("add", "x", *self.flags({"--branch": "a..b"})),
("add", "x", *self.flags({"--status": "done"})),
("add", "x", *self.flags({"--status": None})),
("add", "x", *self.flags({"--pr": "0"})),
("add", "x", *self.flags({"--target": "a b"})),
("update", "none", "--status", "pr"),
("update", "db", "--status", "merged", "--sha", "zzz"),
("update", "db", "--status", "merged"),
("update", "db", "--status", "pr", "--repo", "seed"),
("update", "db"),
("pending", "--repo", str(self.base / "absent")),
("pending", "--repo", str(self.base / "home")),
("match", "--line", "x", "--repo", str(self.base / "absent")),
("void", "--repo", str(self.work)),
("match",),
("sync", "--repo", str(self.work), "--task", "work/none"),
):
with self.subTest(args=args):
self.assertEqual(self.hotfix(*args).returncode, 2)
self.assertEqual(register.read_text(encoding="utf-8"), before)
self.rasci("set", "work/twin", "--branch", "line", "--base", "feature", "--workdir", str(self.feature))
result = self.sync(self.work, "work/side")
self.assertEqual(result.returncode, 2)
self.assertIn("branch line has several bases in the register: work/line=main, work/twin=feature", result.stderr)
if __name__ == "__main__":
unittest.main()
tests/test_rasci.py
+31
View File
@@ -151,6 +151,15 @@ class TestRasci(unittest.TestCase):
self._run("set", "work/a", "--status", "running")
self.assertFalse(any(line.startswith("end:") for line in self._out("get", "work/a")))
def test_base_names_the_upstream_branch_after_the_branch(self):
self._run("set", "work/side", "--branch", "sidetree", "--base", "line")
lines = self._out("get", "work/side")
self.assertIn("base: line", lines)
self.assertEqual(lines.index("base: line"), lines.index("branch: sidetree") + 1)
self.assertEqual(json.loads((self.root / "rasci.json").read_text(encoding="utf-8"))["work/side"]["base"], "line")
self._run("set", "work/side", "--base", "")
self.assertFalse(any(line.startswith("base:") for line in self._out("get", "work/side")))
def test_done_tasks_show_only_with_all(self):
self._run("set", "work/a", "--status", "done", "--end", "2026-11-01T12:00:00+00:00")
self._run("role", "work/a", "Skills", "RA")
@@ -279,6 +288,28 @@ class TestRasci(unittest.TestCase):
self._run("estimate", "work/b", "One", "--drop")
self.assertFalse(any(line.startswith("estimate ") for line in self._out("get", "work/b")))
def test_hotfix_and_operator_rows_are_no_work_tasks_for_gaps_and_wsjf(self):
self._run("set", "work/a")
self._run("role", "work/a", "Skills", "RA")
self._run("kr", "work/a", "add", "released")
self._run("set", "hotfix/db", "--status", "running")
self._run("set", "operator/push", "--status", "running")
shown = self._out("show")
self.assertEqual([line.split(" | ")[2] for line in shown[2:]], ["hotfix/db", "operator/push", "work/a"])
self.assertFalse(any(line.startswith("gaps:") for line in shown))
self.assertEqual(
self._out("wsjf")[2:], ["| 1 | work/a | - (0/1 KR) | - | - | - | - | queued |", "unestimated: work/a"]
)
self._run("set", "work/b", "--title", "Beta")
self.assertEqual(self._out("show")[-1], "gaps: work/b has 0 A; work/b has no R; work/b has no key result")
self._run("set", "hotfix/db", "--status", "done", "--branch", "fix/db", "--workdir", "/w/db")
self._run("set", "operator/push", "--branch", "main", "--artifact", "dir:/tmp/push")
self._run("set", "work/a", "--branch", "feat/a")
for args in (("artifacts",), ("artifacts", "--ended"), ("artifacts", "--task", "hotfix/db")):
with self.subTest(args=args):
self.assertEqual([line.split("\t")[1] for line in self._out(*args)],
["work/a"] if args == ("artifacts",) else [])
def test_order_pins_the_agents_in_the_operators_order(self):
self._run("set", "work/a1", "--worker", "Alpha", "--prio", "1")
self._run("set", "work/a2", "--worker", "Alpha", "--prio", "2")