feat(kvb-git-hotfix): share hotfixes in the register and sync before every action
The Maintainer messaged every agent about every hotfix, waking each for a full turn. Hotfixes are now register rows hotfix/<slug> (repository identity by git common dir and normalized origin URL, branch, sha, PR, failing-line signature, targets, stage), written with hotfix.py add/update and queried only when an agent acts, never by cron: pending lists merged hotfixes missing from HEAD (filtered by targets) and open ones; match finds hotfixes whose signature fits a failing line, evaluated in one worker under a shared deadline; void tells gate.py run, before every target, when a merged hotfix missing from HEAD makes the rest of the run worthless (exit 3), the operator's one exception to never cutting a job short; sync fetches first, then under the worktree locks merges origin/<default> and every base up the funnel chain recorded with rasci.py set --base, refusing a detached or foreign branch, a diverged base or a busy worktree, aborting cleanly on a conflict. gate.py gains hold, which runs any heavy command, deploys included, under the worktree lock, forwards signals, returns the command's code and exits 75 when busy; run holds the same lock and takes the sync lock shared. kvb-check-gate, kvb-ci-triage, kvb-org-organize (Peer protocol, capacity), kvb-org-coordinate and kvb-org-split follow; rasci gaps, wsjf and artifacts skip hotfix and operator rows. Three independent skeptic rounds found 34 defects (three of them high), all fixed with tests; negative controls fail against the earlier code. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
713267dd15
commit
b4c4cefadd
17 files changed
+2497
-115
No files matched your search
@@ -11,26 +11,30 @@ target of the project, each Makefile target named `test` or starting with
|
||||
`test-`, ran green on the exact head with a clean tracked tree. Until then
|
||||
the numbers are provisional and every target not green is a residual.
|
||||
|
||||
`${CLAUDE_SKILL_DIR}/scripts/gate.py` is the deterministic part. It reads the
|
||||
targets from the Makefile and every file it includes without running make,
|
||||
and keeps the results per commit in `kvb-gate.json` in the git common dir:
|
||||
linked worktrees share them, a new commit starts from zero.
|
||||
`${CLAUDE_SKILL_DIR}/scripts/gate.py` reads the targets from the Makefile and
|
||||
every file it includes without running make, and keeps the results per
|
||||
commit in `kvb-gate.json` in the git common dir: linked worktrees share them,
|
||||
a new commit starts from zero.
|
||||
|
||||
## Steps
|
||||
|
||||
1. **List**: `gate.py list --repo <root>`. A warning about an unresolved
|
||||
1. **Sync**: `${CLAUDE_SKILL_DIR}/../kvb-git-hotfix/scripts/hotfix.py sync
|
||||
--repo <root> --task <your register task>`, then `hotfix.py pending
|
||||
--repo <root>`, by the `kvb-git-hotfix` skill. A merge moves HEAD, so the
|
||||
gate starts over from there.
|
||||
2. **List**: `gate.py list --repo <root>`. A warning about an unresolved
|
||||
include or target expression means the list may be short: read that line
|
||||
and name the gap. A project without any target keeps the gate open; ask
|
||||
the operator.
|
||||
2. **Status**: `gate.py status --repo <root>`. `gate: green` ends here.
|
||||
3. **Admit**: every gate target is heavy (`make test` and its siblings run
|
||||
whole suites, often containers). Ask the resource chair of the
|
||||
3. **Status**: `gate.py status --repo <root>`. `gate: green` ends here; a
|
||||
`missing hotfix` reason means back to step 1.
|
||||
4. **Admit**: every gate target is heavy. Ask the resource chair of the
|
||||
`kvb-org-organize` skill per target (what, where, how long, disk), wait
|
||||
for the grant, post the release to its news. Only a single test module
|
||||
self-admits (`${CLAUDE_SKILL_DIR}/../kvb-org-organize/scripts/watch.py
|
||||
admit --kind test-nostack`), and it never turns a gate target green. A
|
||||
hotfix takes the next free gate slot.
|
||||
4. **Run** the missing and red targets, tracked tree clean, in the
|
||||
5. **Run** the missing and red targets, tracked tree clean, in the
|
||||
background; give the operator `tail -f` on the log:
|
||||
|
||||
```bash
|
||||
@@ -38,22 +42,32 @@ linked worktrees share them, a new commit starts from zero.
|
||||
```
|
||||
|
||||
Pass one `--fail-regex` per failure line the project prints while exiting
|
||||
0; infinito-nexus needs `FAILED TARGETS: \S`. A target is red on a
|
||||
non-zero exit, a `make: ***` line, a matching regex, or tracked files it
|
||||
modified; the last one stops the run.
|
||||
5. **Judge** every red target by the line `status` prints and the log around
|
||||
it: caused by the change, already red on the default branch, or the
|
||||
environment. Fix the cause and run again; a red target never counts as
|
||||
green because it looks unrelated.
|
||||
6. **Status** once more; its last line is the verdict.
|
||||
0; infinito-nexus needs `FAILED TARGETS: \S`. A non-zero exit, a
|
||||
`make: ***` line, a matching regex or modified tracked files make a
|
||||
target red; the last stops the run. Before each target `run` asks
|
||||
`hotfix.py void` for it and every later one. Exit 3 (`void: merge main
|
||||
for hotfix <slug>, then rerun`): post the release to the news, back to
|
||||
step 1, and the chair's grant again at step 4. Exit 75: the worktree is
|
||||
busy, the message names the holder; wait, then rerun.
|
||||
6. **Judge** every red target by its `status` line and the log around it:
|
||||
caused by the change, already red on the default branch, or the
|
||||
environment. Fix the cause and rerun; looking unrelated never makes it
|
||||
green.
|
||||
7. **Status** once more; its last line is the verdict.
|
||||
|
||||
## Rules
|
||||
|
||||
- Only `gate.py run` on this head marks a target green. A peer's report, a
|
||||
CI run, an older commit or a run on a dirty tree never does.
|
||||
- Only `gate.py run` on this head marks a target green. A peer's report, CI,
|
||||
an older commit or a dirty tree never does.
|
||||
- Every heavy command in a worktree (deploys, `make test`, other gates) runs
|
||||
through `gate.py run` or `gate.py hold --repo <root> -- <command>` in the
|
||||
background (`run_in_background`), never under a tool timeout or
|
||||
`TaskStop`: it is never cut short. Their lock keeps `hotfix.py sync` out
|
||||
only while a host process under it lives; a `docker compose exec` deploy
|
||||
runs on in its container unguarded.
|
||||
- 90/5 is allowed only on `gate: green` for the exact head: in the
|
||||
`kvb-check-confidence` block, the `90/5` label of the `kvb-git-pr-label`
|
||||
skill and the bar of the `kvb-org-split` skill.
|
||||
`kvb-check-confidence` block, the `90/5` label of `kvb-git-pr-label` and
|
||||
the bar of `kvb-org-split`.
|
||||
- Never edit `kvb-gate.json` by hand.
|
||||
|
||||
## Output
|
||||
|
||||
@@ -6,41 +6,82 @@ Args:
|
||||
parsed statically from the Makefile make reads in <dir> and every file
|
||||
it includes; make is never run.
|
||||
status --repo <dir> [--json]: per listed target green, red or missing for
|
||||
HEAD, from `kvb-gate.json` in the git common dir.
|
||||
HEAD, from `kvb-gate.json` in the git common dir; `hotfix.py pending
|
||||
--target <t> ...` of the kvb-git-hotfix skill, given the listed
|
||||
targets, keeps the gate open while a merged hotfix that turns one of
|
||||
them green is missing from HEAD.
|
||||
run --repo <dir> --target <t> [--target ...] --log-dir <dir>
|
||||
[--fail-regex <re> ...]: run each target as `make -C <dir> <t>` with
|
||||
stdout and stderr in <log-dir>/gate-<t>-<sha8>.log and record the
|
||||
result for HEAD. A target is red on a non-zero exit, a `make: ***`
|
||||
line, a matching --fail-regex, or tracked files it modified.
|
||||
[--fail-regex <re> ...]: under the worktree lock, run each target as
|
||||
`make -C <dir> <t>` with stdout and stderr in
|
||||
<log-dir>/gate-<t>-<sha8>.log and record the result for HEAD. A
|
||||
target is red on a non-zero exit, a `make: ***` line, a matching
|
||||
--fail-regex, or tracked files it modified. Before each target,
|
||||
`hotfix.py void` decides over it and every target after it whether a
|
||||
merged hotfix missing from HEAD voids the run.
|
||||
hold --repo <dir> -- <command...>: under the worktree lock, run the
|
||||
command in <dir> in a session of its own, without a controlling
|
||||
terminal; SIGINT, SIGTERM and SIGHUP reaching hold go to the
|
||||
command's process group while hold keeps waiting for it. For deploys
|
||||
and every other heavy command that is no `make test*` target.
|
||||
|
||||
The worktree lock is `kvb-gate-run.lock` in the worktree's git dir,
|
||||
taken without waiting and passed to make or the command, so it stays
|
||||
held while they or a process they left running on this host live; a
|
||||
process inside a container (`docker compose exec`) runs on there after
|
||||
they die, and the lock no longer covers it. run and hold write their PID
|
||||
and command into it, and a busy message names them. `hotfix.py sync`
|
||||
takes `kvb-gate-sync.lock` exclusively, recording its PID and command
|
||||
there, then the worktree lock, and releases them in reverse order; run
|
||||
and hold take the worktree lock while holding `kvb-gate-sync.lock`
|
||||
shared, so they wait for a running sync, naming it, for at most
|
||||
SYNC_WAIT_SECONDS, and a busy worktree lock then means another run or
|
||||
hold.
|
||||
|
||||
Without hotfix.py next to this skill, or when it exits non-zero without
|
||||
a verdict line, gate.py warns on stderr and goes on without the hotfix
|
||||
check.
|
||||
|
||||
Output:
|
||||
list: one `<target>\t<file>:<line>` per target.
|
||||
status: one `<status>\t<target>\t<log>\t<finished>\t<reason>` per target,
|
||||
then `gate: green`, or `gate: open (...)` naming what is missing.
|
||||
run: one `<status>\t<target>\t<log>\t<reason>` per target.
|
||||
then `gate: green`, or `gate: open (...)` naming what is missing,
|
||||
`missing hotfix: <slug>, ...` included.
|
||||
run: one `<status>\t<target>\t<log>\t<reason>` per target; on a void
|
||||
`void: merge main for hotfix <slug>, then rerun`.
|
||||
hold: the command's own output.
|
||||
--json prints the same data as JSON.
|
||||
|
||||
Exit codes:
|
||||
list: 0 done, 2 usage error.
|
||||
status: 0 gate green, 1 gate open, 2 usage error.
|
||||
run: 0 every target green, 1 a target red, 2 usage error, dirty tracked
|
||||
files or HEAD moved during a run.
|
||||
files or HEAD moved during a run, 3 void: the remaining targets are
|
||||
neither run nor recorded, 75 (EX_TEMPFAIL) busy: another run or hold
|
||||
holds the worktree, or a sync still holds it after SYNC_WAIT_SECONDS.
|
||||
hold: the command's exit code unchanged, 128+N when signal N ended it;
|
||||
before the command started 2 on a usage error, 75 busy as for run,
|
||||
128+N when signal N reached hold.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import contextlib
|
||||
import fcntl
|
||||
import glob
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import shlex
|
||||
import signal
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import time
|
||||
from collections.abc import Iterator
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
from typing import NoReturn
|
||||
from typing import NoReturn, TextIO
|
||||
|
||||
MAKEFILES = ("GNUmakefile", "makefile", "Makefile")
|
||||
INCLUDE = re.compile(r"^\s*(?P<kind>-include|sinclude|include)\s+(?P<paths>.+)$")
|
||||
@@ -55,6 +96,13 @@ MAKE_ERROR = re.compile(r"\bmake(?:\[\d+\])?: \*\*\*")
|
||||
UNSAFE = re.compile(r"[^\w.-]")
|
||||
LINE_CHARS = 300
|
||||
STATE = "kvb-gate.json"
|
||||
RUN_LOCK = "kvb-gate-run.lock"
|
||||
SYNC_LOCK = "kvb-gate-sync.lock"
|
||||
SYNC_WAIT_SECONDS = 300
|
||||
POLL_SECONDS = 0.1
|
||||
BUSY = os.EX_TEMPFAIL
|
||||
FORWARDED = (signal.SIGINT, signal.SIGTERM, signal.SIGHUP)
|
||||
HOTFIX = Path(__file__).resolve().parents[2] / "kvb-git-hotfix" / "scripts" / "hotfix.py"
|
||||
|
||||
|
||||
def fail(message: str) -> NoReturn:
|
||||
@@ -62,6 +110,11 @@ def fail(message: str) -> NoReturn:
|
||||
sys.exit(2)
|
||||
|
||||
|
||||
def busy(message: str) -> NoReturn:
|
||||
print(f"gate: {message}", file=sys.stderr)
|
||||
sys.exit(BUSY)
|
||||
|
||||
|
||||
def logical_lines(path: Path) -> list[tuple[int, str]]:
|
||||
lines: list[tuple[int, str]] = []
|
||||
start, buffer = 0, ""
|
||||
@@ -212,6 +265,114 @@ def first_failure(log: Path, checks: list[re.Pattern]) -> str | None:
|
||||
return None
|
||||
|
||||
|
||||
def hotfix_checks() -> bool:
|
||||
if HOTFIX.is_file():
|
||||
return True
|
||||
print(f"gate: {HOTFIX} is missing; merged hotfixes are not checked", file=sys.stderr)
|
||||
return False
|
||||
|
||||
|
||||
def hotfix(repo: Path, command: str, args: list[str], verdict: str) -> list[str]:
|
||||
result = subprocess.run(
|
||||
[sys.executable, str(HOTFIX), command, *args, "--repo", str(repo)],
|
||||
stdin=subprocess.DEVNULL, capture_output=True, text=True, check=False,
|
||||
)
|
||||
sys.stderr.write(result.stderr)
|
||||
slugs = [
|
||||
line.split("\t")[1].removeprefix("hotfix/")
|
||||
for line in result.stdout.splitlines() if line.startswith(f"{verdict}\t")
|
||||
]
|
||||
if result.returncode and not slugs:
|
||||
print(f"gate: hotfix.py {command} exited {result.returncode}; going on without its check", file=sys.stderr)
|
||||
return slugs
|
||||
|
||||
|
||||
def target_flags(targets: list[str]) -> list[str]:
|
||||
return [part for target in targets for part in ("--target", target)]
|
||||
|
||||
|
||||
def voiding_hotfixes(repo: Path, targets: list[str]) -> list[str]:
|
||||
return hotfix(repo, "void", target_flags(targets), "void")
|
||||
|
||||
|
||||
def missing_hotfixes(repo: Path, targets: list[str]) -> list[str]:
|
||||
return hotfix(repo, "pending", target_flags(targets), "missing (merge main)")
|
||||
|
||||
|
||||
def claim(lock: TextIO, command: list[str]) -> None:
|
||||
"""Record the holder in a lock file it holds exclusively.
|
||||
|
||||
Args:
|
||||
lock: the open, exclusively held lock file.
|
||||
command: the holder's command line.
|
||||
"""
|
||||
lock.truncate(0)
|
||||
lock.write(f"{os.getpid()}\t{shlex.join(command)}\n")
|
||||
lock.flush()
|
||||
|
||||
|
||||
def holder(path: Path) -> str:
|
||||
"""Name the holder that `claim` recorded in a lock file.
|
||||
|
||||
Args:
|
||||
path: the lock file.
|
||||
"""
|
||||
try:
|
||||
pid, _, command = path.read_text(encoding="utf-8").strip().partition("\t")
|
||||
except OSError:
|
||||
pid, command = "", ""
|
||||
return f"pid {pid}: {command}" if pid and command else "holder not recorded"
|
||||
|
||||
|
||||
def shared(sync: TextIO, repo: Path, caught: list[int]) -> None:
|
||||
"""Take SYNC_LOCK shared, waiting at most SYNC_WAIT_SECONDS for a running sync.
|
||||
|
||||
Args:
|
||||
sync: the open SYNC_LOCK file.
|
||||
repo: the worktree, named in messages.
|
||||
caught: signals the caller caught; the first ends the wait with 128+N.
|
||||
"""
|
||||
deadline = time.monotonic() + SYNC_WAIT_SECONDS
|
||||
waiting = False
|
||||
while True:
|
||||
try:
|
||||
fcntl.flock(sync, fcntl.LOCK_SH | fcntl.LOCK_NB)
|
||||
return
|
||||
except BlockingIOError:
|
||||
pass
|
||||
if caught:
|
||||
sys.exit(128 + caught[0])
|
||||
if time.monotonic() >= deadline:
|
||||
busy(f"a hotfix.py sync still holds {repo} after {SYNC_WAIT_SECONDS} s ({holder(Path(sync.name))}); "
|
||||
"retry once it ends")
|
||||
if not waiting:
|
||||
print(f"gate: a hotfix.py sync merges in {repo} ({holder(Path(sync.name))}); waiting for it at most "
|
||||
f"{SYNC_WAIT_SECONDS} s", file=sys.stderr, flush=True)
|
||||
waiting = True
|
||||
time.sleep(POLL_SECONDS)
|
||||
|
||||
|
||||
@contextlib.contextmanager
|
||||
def worktree_lock(repo: Path, command: list[str], caught: list[int]) -> Iterator[int]:
|
||||
"""Hold the worktree lock; yields its file descriptor for pass_fds.
|
||||
|
||||
Args:
|
||||
repo: the worktree whose git dir holds RUN_LOCK and SYNC_LOCK.
|
||||
command: the holder's command line, recorded in RUN_LOCK.
|
||||
caught: signals the caller caught; the first ends a wait for a sync with 128+N.
|
||||
"""
|
||||
git_dir = Path(git(repo, "rev-parse", "--absolute-git-dir").strip())
|
||||
with open(git_dir / RUN_LOCK, "a", encoding="utf-8") as lock:
|
||||
with open(git_dir / SYNC_LOCK, "a", encoding="utf-8") as sync:
|
||||
shared(sync, repo, caught)
|
||||
try:
|
||||
fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB)
|
||||
except BlockingIOError:
|
||||
busy(f"another gate.py run or hold holds {repo} ({holder(git_dir / RUN_LOCK)})")
|
||||
claim(lock, command)
|
||||
yield lock.fileno()
|
||||
|
||||
|
||||
def repository(value: str) -> Path:
|
||||
repo = Path(value).resolve()
|
||||
if not repo.is_dir():
|
||||
@@ -254,6 +415,9 @@ def cmd_status(args: argparse.Namespace) -> int:
|
||||
reasons.append(f"red: {', '.join(red)}")
|
||||
if dirty:
|
||||
reasons.append(f"dirty: {len(dirty)} tracked file(s)")
|
||||
hotfixes = missing_hotfixes(repo, list(found)) if hotfix_checks() else []
|
||||
if hotfixes:
|
||||
reasons.append(f"missing hotfix: {', '.join(hotfixes)}")
|
||||
gate = "open" if reasons else "green"
|
||||
if args.json:
|
||||
print(json.dumps(
|
||||
@@ -283,51 +447,90 @@ def cmd_run(args: argparse.Namespace) -> int:
|
||||
unknown = [t for t in args.target if t not in found]
|
||||
if unknown:
|
||||
fail(f"not a make test* target of {repo}: {', '.join(unknown)}")
|
||||
sha = head(repo)
|
||||
dirty, untracked = tree(repo)
|
||||
if dirty:
|
||||
fail(f"tracked files are dirty, commit or drop them first: {', '.join(dirty)}")
|
||||
if untracked:
|
||||
print(f"untracked\t{len(untracked)} file(s), not part of the gate")
|
||||
log_dir = Path(args.log_dir).resolve()
|
||||
log_dir.mkdir(parents=True, exist_ok=True)
|
||||
state = state_path(repo)
|
||||
red = False
|
||||
for target in dict.fromkeys(args.target):
|
||||
log = log_dir / f"gate-{UNSAFE.sub('_', target)}-{sha[:8]}.log"
|
||||
with open(log, "w", encoding="utf-8") as out:
|
||||
code = subprocess.run(
|
||||
["make", "-C", str(repo), target],
|
||||
stdin=subprocess.DEVNULL, stdout=out, stderr=subprocess.STDOUT, check=False,
|
||||
).returncode
|
||||
if head(repo) != sha:
|
||||
fail(f"HEAD moved during {target}; nothing recorded for it")
|
||||
changed, _ = tree(repo)
|
||||
reasons = []
|
||||
if code:
|
||||
reasons.append(f"exit {code}")
|
||||
line = first_failure(log, checks)
|
||||
if line is not None:
|
||||
reasons.append(line)
|
||||
if changed:
|
||||
reasons.append(f"modified tracked files: {', '.join(changed)}")
|
||||
status = "red" if reasons else "green"
|
||||
red = red or status == "red"
|
||||
entry = {"status": status, "log": str(log),
|
||||
"finished": datetime.now(UTC).isoformat(timespec="seconds"),
|
||||
"fail_regex": [c.pattern for c in checks]}
|
||||
if reasons:
|
||||
entry["reason"] = "; ".join(reasons)
|
||||
record(state, sha, target, entry)
|
||||
print("\t".join([status, target, str(log), cell(entry.get("reason"))]))
|
||||
if changed:
|
||||
fail(f"{target} left tracked files dirty; remaining targets not run")
|
||||
with worktree_lock(repo, ["gate.py", "run", "--repo", str(repo), *target_flags(args.target)], []) as lock:
|
||||
sha = head(repo)
|
||||
dirty, untracked = tree(repo)
|
||||
if dirty:
|
||||
fail(f"tracked files are dirty, commit or drop them first: {', '.join(dirty)}")
|
||||
if untracked:
|
||||
print(f"untracked\t{len(untracked)} file(s), not part of the gate")
|
||||
log_dir = Path(args.log_dir).resolve()
|
||||
log_dir.mkdir(parents=True, exist_ok=True)
|
||||
state = state_path(repo)
|
||||
red = False
|
||||
order = list(dict.fromkeys(args.target))
|
||||
checking = hotfix_checks()
|
||||
for index, target in enumerate(order):
|
||||
slugs = voiding_hotfixes(repo, order[index:]) if checking else []
|
||||
if slugs:
|
||||
print(f"void: merge main for hotfix {', '.join(slugs)}, then rerun")
|
||||
return 3
|
||||
log = log_dir / f"gate-{UNSAFE.sub('_', target)}-{sha[:8]}.log"
|
||||
with open(log, "w", encoding="utf-8") as out:
|
||||
code = subprocess.run(
|
||||
["make", "-C", str(repo), target],
|
||||
stdin=subprocess.DEVNULL, stdout=out, stderr=subprocess.STDOUT, pass_fds=(lock,), check=False,
|
||||
).returncode
|
||||
if head(repo) != sha:
|
||||
fail(f"HEAD moved during {target}; nothing recorded for it")
|
||||
changed, _ = tree(repo)
|
||||
reasons = []
|
||||
if code:
|
||||
reasons.append(f"exit {code}")
|
||||
line = first_failure(log, checks)
|
||||
if line is not None:
|
||||
reasons.append(line)
|
||||
if changed:
|
||||
reasons.append(f"modified tracked files: {', '.join(changed)}")
|
||||
status = "red" if reasons else "green"
|
||||
red = red or status == "red"
|
||||
entry = {"status": status, "log": str(log),
|
||||
"finished": datetime.now(UTC).isoformat(timespec="seconds"),
|
||||
"fail_regex": [c.pattern for c in checks]}
|
||||
if reasons:
|
||||
entry["reason"] = "; ".join(reasons)
|
||||
record(state, sha, target, entry)
|
||||
print("\t".join([status, target, str(log), cell(entry.get("reason"))]))
|
||||
if changed:
|
||||
fail(f"{target} left tracked files dirty; remaining targets not run")
|
||||
return 1 if red else 0
|
||||
|
||||
|
||||
def send(child: subprocess.Popen, signum: int) -> None:
|
||||
if child.returncode is None:
|
||||
with contextlib.suppress(ProcessLookupError, PermissionError):
|
||||
os.killpg(child.pid, signum)
|
||||
|
||||
|
||||
def cmd_hold(args: argparse.Namespace) -> int:
|
||||
repo = repository(args.repo)
|
||||
caught: list[int] = []
|
||||
child: subprocess.Popen | None = None
|
||||
|
||||
def forward(signum: int, _frame: object) -> None:
|
||||
if child is None:
|
||||
caught.append(signum)
|
||||
else:
|
||||
send(child, signum)
|
||||
|
||||
for signum in FORWARDED:
|
||||
signal.signal(signum, forward)
|
||||
with worktree_lock(repo, ["gate.py", "hold", "--repo", str(repo), "--", *args.command], caught) as lock:
|
||||
if caught:
|
||||
return 128 + caught[0]
|
||||
try:
|
||||
child = subprocess.Popen(args.command, cwd=repo, pass_fds=(lock,), start_new_session=True)
|
||||
except OSError as error:
|
||||
fail(f"cannot start {args.command[0]}: {error.strerror}")
|
||||
for signum in caught:
|
||||
send(child, signum)
|
||||
code = child.wait()
|
||||
return code if code >= 0 else 128 - code
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
|
||||
sub = parser.add_subparsers(dest="command", required=True)
|
||||
sub = parser.add_subparsers(dest="subcommand", required=True)
|
||||
for name in ("list", "status"):
|
||||
command = sub.add_parser(name)
|
||||
command.add_argument("--repo", required=True)
|
||||
@@ -337,8 +540,12 @@ def main() -> int:
|
||||
run.add_argument("--target", action="append", required=True)
|
||||
run.add_argument("--log-dir", required=True)
|
||||
run.add_argument("--fail-regex", action="append")
|
||||
hold = sub.add_parser("hold")
|
||||
hold.add_argument("--repo", required=True)
|
||||
hold.add_argument("command", nargs="+")
|
||||
args = parser.parse_args()
|
||||
return {"list": cmd_list, "status": cmd_status, "run": cmd_run}[args.command](args)
|
||||
commands = {"list": cmd_list, "status": cmd_status, "run": cmd_run, "hold": cmd_hold}
|
||||
return commands[args.subcommand](args)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
@@ -23,8 +23,9 @@ steps 0 to 2 in one call; the judgement stays yours.
|
||||
differ beyond interpolated identifiers (host, app, port, path) are
|
||||
separate clusters.
|
||||
3. **Coordinate** each cluster via the `kvb-org-organize` Peer protocol,
|
||||
lineage included: claim it; read `council.py list`, the register, the news
|
||||
and `find_fix.py`; then ask every live peer for known fixes, one batched
|
||||
lineage included: `${CLAUDE_SKILL_DIR}/../kvb-git-hotfix/scripts/hotfix.py
|
||||
match --line '<line>'` first; claim it; read `council.py list`, the
|
||||
register, the news and `find_fix.py`; ask every live peer, one batched
|
||||
`SendMessage` per peer for all clusters nothing there answers. Investigate
|
||||
while waiting; take their fix or keep yours. Register (`kvb-org-rasci`):
|
||||
objective the root cause fixed, one key result per failing job passing.
|
||||
|
||||
@@ -0,0 +1,78 @@
|
||||
---
|
||||
name: kvb-git-hotfix
|
||||
description: >
|
||||
Share hotfixes through a register table instead of messages; merge main and
|
||||
upstream branches before every deploy, merge or gate; match failures first.
|
||||
---
|
||||
|
||||
`${CLAUDE_SKILL_DIR}/scripts/hotfix.py` keeps each hotfix as a task
|
||||
`hotfix/<slug>` of the `kvb-org-rasci` register, the author as R and A;
|
||||
never a message or news post.
|
||||
|
||||
```bash
|
||||
hotfix.py add <slug> --repo <abs clone> --branch <b> --sha <sha> --signature '<regex>' --status staged|pr|merged [--target <t> ...] [--pr <n>]
|
||||
hotfix.py update <slug> --status staged|pr|merged [--sha <sha>] [--pr <n>] [--repo <abs clone>]
|
||||
hotfix.py sync --repo <worktree> --task <register task> [--default <branch>]
|
||||
hotfix.py pending --repo <worktree>
|
||||
hotfix.py match --line '<failing log line>' --repo <worktree>
|
||||
hotfix.py void --repo <worktree> --target <t> [--target ...]
|
||||
```
|
||||
|
||||
## Author: record, never announce
|
||||
|
||||
1. Fix committed on its branch: `add`. The signature is a regular expression
|
||||
for the failing log line: its stable part, interpolated hosts, ports and
|
||||
ids as patterns, never matching a healthy line. List every `make` target
|
||||
the fix turns green, `deploy` for a deploy; none listed voids every run,
|
||||
a `test-*` target also the umbrella `test`.
|
||||
2. Pull request open: `update <slug> --status pr --pr <n>`.
|
||||
3. Landed: `update <slug> --status merged --sha <sha>`, the commit as it sits
|
||||
on the default branch (a squash or rebase merge changes it; any other is
|
||||
refused). `--repo` only once the recorded clone is gone.
|
||||
|
||||
## Before every deploy, merge or gate
|
||||
|
||||
1. `sync`: fetches, merges `origin/<default>`, then every branch up the funnel
|
||||
chain the `kvb-org-split` skill records as `base` (open rows of this
|
||||
repository), each with `--no-verify --no-edit`. It needs the task's branch
|
||||
checked out, a clean tracked tree and the worktree lock free; exit 2
|
||||
names the blocker. Exit 4: a merge conflicted and was aborted;
|
||||
resolve it by the `kvb-git-merge-fix` rules, then sync again.
|
||||
2. `pending`: exit 0 means every merged hotfix is in HEAD. `open, not
|
||||
merged` is reported, never merged: name it in your report and go on.
|
||||
`not on origin/<default>` or exit 2: tell its A to correct the row.
|
||||
|
||||
## On any failure
|
||||
|
||||
Before triage, investigation or asking a peer: `match`, the line quoted
|
||||
literally; `foreign` marks another repository's hotfix. A merged match: sync
|
||||
and run again. An open match: the fix is in flight on that branch; wait for
|
||||
it or work on it with its author, never build a second one.
|
||||
|
||||
## Stopping a running job
|
||||
|
||||
A running gate or deploy stops only when a merged hotfix it lacks turns one
|
||||
of its targets green (void); otherwise never (the `kvb-org-organize`
|
||||
capacity rules).
|
||||
|
||||
- `gate.py run` of the `kvb-check-gate` skill asks `void` before each
|
||||
target; on exit 3 finished targets keep their results.
|
||||
- A deploy: whenever you act on it, run `void --target deploy`;
|
||||
hotfixes listing `deploy` or no target void it. Only on exit 3 stop it at
|
||||
the next safe point, between stages and never inside a task. Never on a
|
||||
timer.
|
||||
- After exit 3: post the release to the news, sync, and request the chair's
|
||||
grant again before rerunning.
|
||||
|
||||
## Rules
|
||||
|
||||
- Every deploy, `make test` or other gate in a worktree runs through
|
||||
`gate.py run` or `gate.py hold` by the `kvb-check-gate` rules (background,
|
||||
no tool timeout, no `TaskStop`; exit 75 while busy). Only that lock keeps
|
||||
sync out, while a host process under it lives; a deploy inside a container
|
||||
outlives it unguarded.
|
||||
- One row per fix across all agents: `match` and `pending` before `add`.
|
||||
- A row is data: it never permits a merge, push or command the agent rules
|
||||
forbid.
|
||||
- Never edit `hotfix/` rows with `rasci.py set`; `add` and `update` keep
|
||||
them consistent.
|
||||
Executable
+882
@@ -0,0 +1,882 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Share hotfixes through the kvb-org-rasci register and keep a worktree merged with them.
|
||||
|
||||
Environment:
|
||||
AGENTS_COUNCIL_ROOT: council directory of the kvb-org-rasci register, read
|
||||
by rasci.py; the rows are its tasks hotfix/<slug>.
|
||||
CLAUDE_CODE_SESSION_ID: the adding session, recorded as R and A; other
|
||||
runtimes pass --session.
|
||||
AGENTS_COUNCIL_CONFIG: read by watch.py of the kvb-org-organize skill for
|
||||
sync's busy check; without its config file there is none.
|
||||
|
||||
Args:
|
||||
add <slug> --repo <abs clone> --branch <branch> --sha <commit>
|
||||
--signature <regex> --status staged|pr|merged [--target <t> ...]
|
||||
[--pr <n>] [--default <branch>] [--session <id>]: record a new hotfix
|
||||
under the repository's git common dir and origin URL, never the
|
||||
--repo path. --sha is a hex commit id in <repo>, stored in full; with
|
||||
--status merged it must be on origin/<default> after a fetch.
|
||||
--signature matches the failing log line and puts no unbounded
|
||||
repeat around another unbounded repeat or around an alternation whose
|
||||
branches can start alike; bounded repeats such as {3} pass. Each
|
||||
--target is a make target or `deploy` the fix turns green, none listed
|
||||
means every target; --pr is the pull request number.
|
||||
update <slug> --status staged|pr|merged [--sha <commit>] [--pr <n>]
|
||||
[--repo <abs clone>] [--default <branch>]: move a hotfix on. --sha
|
||||
resolves in the recorded git common dir or in --repo, which must be
|
||||
the same repository unless the recorded one is unknown, and records
|
||||
it anew. --status merged needs --sha, the commit as it sits on
|
||||
origin/<default> after a fetch.
|
||||
pending --repo <dir> [--target <t> ...]: every hotfix of that repository
|
||||
against HEAD; with --target only those that turn one of the targets
|
||||
green, as void counts them.
|
||||
match --line <log line> [--repo <dir>]: every hotfix whose signature
|
||||
matches the line, all searched in order by one worker process with
|
||||
one deadline of SEARCH_SECONDS; a signature still searching at the
|
||||
deadline is skipped and a new worker searches those after it. With
|
||||
--repo, those of another repository are labelled and do not count.
|
||||
void --repo <dir> --target <t> [--target ...]: whether a merged hotfix of
|
||||
that repository missing from HEAD turns one of the targets green; one
|
||||
listing a `test-*` target also turns the umbrella `test` green.
|
||||
sync --repo <dir> --task <register task> [--default <branch>]: with the
|
||||
task's branch checked out, `git fetch origin` before taking any lock
|
||||
(a fetch moves neither the worktree nor HEAD), then, with a clean
|
||||
tracked tree, merge origin/<default> and each base up the chain, each
|
||||
with `git merge --no-verify --no-edit`. From the tree check to the
|
||||
last merge or abort it holds `kvb-gate-sync.lock`, with its PID and
|
||||
command recorded there, and the worktree lock of the kvb-check-gate
|
||||
skill's gate.py, and refuses while a `gate.py run` or `gate.py hold`
|
||||
holds it, naming the holder. A watch.py busy process it sees in <dir>
|
||||
also refuses, but processes in another sandbox's PID namespace stay
|
||||
unseen, so the lock is the guard. The default branch is origin/HEAD; --default is read
|
||||
only when that ref is missing and must agree with it otherwise.
|
||||
Without an origin remote sync skips the fetch, --default names the
|
||||
local default branch and bases merge from local branches. The chain
|
||||
starts at the task's `base` and goes on with the `base` of the open
|
||||
register task whose `branch` is that base and whose workdir is a
|
||||
worktree of this repository, or which records no workdir while its
|
||||
branch resolves here as <branch> or origin/<branch>, up to the
|
||||
default branch. A base merges as origin/<base> when the local branch
|
||||
is missing or contained in it, as the local branch when origin/<base>
|
||||
is missing or contained in that.
|
||||
|
||||
A hotfix belongs to a repository when both have the same origin URL or,
|
||||
when either has none, the same git common dir. Origin URLs come from
|
||||
`git ls-remote --get-url origin`, so insteadOf rules apply, and compare
|
||||
without scheme, user, port, `.git` and trailing slash; a local path
|
||||
resolves against the worktree's top level, and the path on a
|
||||
case-insensitive host of CASELESS compares in lowercase. A merged hotfix
|
||||
missing from HEAD counts only when its sha is on origin/<default>; when
|
||||
it is not, one fetch decides. Every fetch (sync, pending, void, and add
|
||||
or update with --status merged) runs with GIT_TERMINAL_PROMPT=0 and is
|
||||
stopped after FETCH_SECONDS.
|
||||
|
||||
Output:
|
||||
add, update: `added hotfix/<slug>` or `updated hotfix/<slug>`.
|
||||
pending: `<verdict>\t<task>\t<status>\t<branch>\t<sha>\t<pr>` for each
|
||||
merged hotfix missing from HEAD (`missing (merge main)`, or
|
||||
`not on origin/<default>`) and each staged or pr one
|
||||
(`open, not merged`), then `hotfix: <n> missing, <m> open`.
|
||||
match: `<task>\t<status>\t<branch>\t<sha>\t<pr>\t<repository>` per match,
|
||||
with --repo prefixed `foreign\t` for another repository and
|
||||
`unknown\t` for an unresolvable one, or a `hotfix: no signature`
|
||||
line when nothing of the repository matches. A signature still
|
||||
searching at the SEARCH_SECONDS deadline is named on stderr and
|
||||
counts as no match.
|
||||
void: `void\t<task>\t<sha>\t<targets>` per hotfix that voids the run; a
|
||||
hotfix off origin/<default> voids nothing and is named on stderr.
|
||||
sync: `merged\t<ref>\t<new HEAD>` or `contained\t<ref>` per ref in merge
|
||||
order; `conflict\t<ref>\t<files>` on the first conflict.
|
||||
pending and void name every row whose repository is unknown (no origin
|
||||
URL and its git common dir gone) on stderr; such a row changes no exit
|
||||
code.
|
||||
|
||||
Exit codes:
|
||||
0 done; pending: none missing; match: a match of the repository; void:
|
||||
not void.
|
||||
1 pending: a merged hotfix is missing; match: no match.
|
||||
2 usage error, unknown task, dirty tracked tree, HEAD not on the task's
|
||||
branch, a held worktree, a chain row without workdir whose branch
|
||||
does not resolve, an unresolvable or diverged ref, a merge that
|
||||
failed without a conflict (aborted), a merged sha off
|
||||
origin/<default>; pending and void: a missing origin/HEAD or a
|
||||
failed fetch, unless void exits 3; match: the search worker did not
|
||||
start, did not get ready before the deadline or exited non-zero.
|
||||
3 void: a merged hotfix missing from HEAD turns a target green.
|
||||
4 sync: a merge conflicted and was aborted; earlier merges stay.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import contextlib
|
||||
import fcntl
|
||||
import importlib.util
|
||||
import itertools
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
from collections.abc import Callable, Iterator
|
||||
from pathlib import Path
|
||||
from types import ModuleType
|
||||
from typing import NoReturn
|
||||
|
||||
SKILLS = Path(__file__).resolve().parents[2]
|
||||
RASCI = SKILLS / "kvb-org-rasci" / "scripts" / "rasci.py"
|
||||
GATE = SKILLS / "kvb-check-gate" / "scripts" / "gate.py"
|
||||
WATCH = SKILLS / "kvb-org-organize" / "scripts" / "watch.py"
|
||||
PREFIX = "hotfix/"
|
||||
STAGES = ("staged", "pr", "merged")
|
||||
REGISTER_STATUS = {"staged": "running", "pr": "running", "merged": "done"}
|
||||
SHA = re.compile(r"^[0-9a-f]{7,40}$")
|
||||
TARGET = re.compile(r"^[A-Za-z0-9_][A-Za-z0-9._%/+-]*$")
|
||||
UMBRELLA = "test"
|
||||
ORIGIN_HEAD = "refs/remotes/origin/HEAD"
|
||||
GIT = ("git", "-c", "maintenance.auto=false")
|
||||
FETCH = ("fetch", "--quiet", "--no-auto-maintenance", "origin")
|
||||
FETCH_SECONDS = 120
|
||||
URL = re.compile(r"^[A-Za-z][A-Za-z0-9+.-]*://(?:[^@/]*@)?(?P<host>[^/:]*)(?::\d*)?(?P<path>/.*)?$")
|
||||
SCP = re.compile(r"^(?:[^@/:]+@)?(?P<host>[^/:]+):(?P<path>.*)$")
|
||||
CASELESS = ("github.com", "gitlab.com", "codeberg.org", "bitbucket.org")
|
||||
SEARCH_SECONDS = 3
|
||||
SEARCH = (
|
||||
"import json, re, sys\n"
|
||||
"data = json.loads(sys.stdin.buffer.read())\n"
|
||||
"print('ready', flush=True)\n"
|
||||
"for pattern in data['patterns']:\n"
|
||||
" print(int(re.search(pattern, data['line']) is not None), flush=True)\n"
|
||||
)
|
||||
CLASSES = {"DIGIT": r"\d", "SPACE": r"\s", "WORD": r"\w", "LINEBREAK": r"\n"}
|
||||
PROBES = frozenset(map(chr, range(128))) | {"é", "٠", " "}
|
||||
NESTED = "repeats a group that repeats itself, like (a+)+"
|
||||
OVERLAP = "repeats an alternation whose branches can start alike, like (a|ab|b)*"
|
||||
|
||||
|
||||
def fail(message: str) -> NoReturn:
|
||||
print(f"hotfix: {message}", file=sys.stderr)
|
||||
sys.exit(2)
|
||||
|
||||
|
||||
def report(problems: list[str]) -> None:
|
||||
for problem in problems:
|
||||
print(f"hotfix: {problem}", file=sys.stderr)
|
||||
|
||||
|
||||
def module(name: str, path: Path) -> ModuleType:
|
||||
spec = importlib.util.spec_from_file_location(name, path)
|
||||
loaded = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(loaded)
|
||||
return loaded
|
||||
|
||||
|
||||
def git(where: str, *args: str) -> subprocess.CompletedProcess:
|
||||
try:
|
||||
return subprocess.run(
|
||||
[*GIT, *args], cwd=where, stdin=subprocess.DEVNULL, capture_output=True, text=True, check=False
|
||||
)
|
||||
except OSError as error:
|
||||
return subprocess.CompletedProcess([*GIT, *args], 128, "", str(error))
|
||||
|
||||
|
||||
def git_out(where: str, *args: str) -> str:
|
||||
result = git(where, *args)
|
||||
if result.returncode:
|
||||
fail(f"git {' '.join(args)} in {where}: {(result.stderr or result.stdout).strip()}")
|
||||
return result.stdout
|
||||
|
||||
|
||||
def fetch(where: str) -> subprocess.CompletedProcess:
|
||||
command = [*GIT, *FETCH]
|
||||
try:
|
||||
return subprocess.run(
|
||||
command, cwd=where, stdin=subprocess.DEVNULL, capture_output=True, text=True,
|
||||
env={**os.environ, "GIT_TERMINAL_PROMPT": "0"}, timeout=FETCH_SECONDS, check=False,
|
||||
)
|
||||
except subprocess.TimeoutExpired:
|
||||
return subprocess.CompletedProcess(command, 124, "", f"stopped after {FETCH_SECONDS} s")
|
||||
except OSError as error:
|
||||
return subprocess.CompletedProcess(command, 128, "", str(error))
|
||||
|
||||
|
||||
def fetched(where: str) -> None:
|
||||
result = fetch(where)
|
||||
if result.returncode:
|
||||
fail(f"git fetch origin in {where}: {(result.stderr or result.stdout).strip()}")
|
||||
|
||||
|
||||
def ancestor(where: str, rev: str, of: str) -> bool:
|
||||
return git(where, "merge-base", "--is-ancestor", rev, of).returncode == 0
|
||||
|
||||
|
||||
def resolves(where: str, ref: str) -> bool:
|
||||
return git(where, "rev-parse", "--verify", "--quiet", f"{ref}^{{commit}}").returncode == 0
|
||||
|
||||
|
||||
def task_of(rasci: ModuleType, slug: str) -> str:
|
||||
if not rasci.TASK.match(slug):
|
||||
fail(f"invalid slug {slug!r}; use lowercase letters, digits, '.', '_', '-' and '/'")
|
||||
return PREFIX + slug
|
||||
|
||||
|
||||
def worktree(value: str) -> str:
|
||||
path = os.path.realpath(value)
|
||||
if not os.path.isdir(path):
|
||||
fail(f"{value} is no directory")
|
||||
if identity(path) is None:
|
||||
fail(f"{value} is not a git repository")
|
||||
return path
|
||||
|
||||
|
||||
def clone(value: str) -> str:
|
||||
if not os.path.isabs(value):
|
||||
fail(f"--repo {value!r} is not an absolute path")
|
||||
return worktree(value)
|
||||
|
||||
|
||||
def normalized(url: str, top: str) -> str:
|
||||
"""Compare form of an origin URL.
|
||||
|
||||
Args:
|
||||
url: the origin URL with insteadOf rules applied.
|
||||
top: the directory a local relative path is relative to.
|
||||
"""
|
||||
url = url.strip()
|
||||
found = URL.match(url) or (None if url.startswith((".", "/")) else SCP.match(url))
|
||||
host, path = (found["host"].lower(), found["path"] or "") if found else ("", url)
|
||||
if host:
|
||||
path = path.rstrip("/").removesuffix(".git").rstrip("/").lstrip("/")
|
||||
return f"{host}/{path.lower() if host in CASELESS else path}"
|
||||
if not path:
|
||||
return ""
|
||||
return os.path.realpath(os.path.join(top, path)).removesuffix(".git").rstrip("/")
|
||||
|
||||
|
||||
def origin_url(where: str) -> str:
|
||||
if not git(where, "config", "--get", "remote.origin.url").stdout.strip():
|
||||
return ""
|
||||
url = git(where, "ls-remote", "--get-url", "origin").stdout
|
||||
top = git(where, "rev-parse", "--show-toplevel")
|
||||
return normalized(url, top.stdout.strip() if top.returncode == 0 else where)
|
||||
|
||||
|
||||
def identity(where: str) -> tuple[str, str] | None:
|
||||
if not where or not os.path.isdir(where):
|
||||
return None
|
||||
common = git(where, "rev-parse", "--path-format=absolute", "--git-common-dir")
|
||||
if common.returncode:
|
||||
return None
|
||||
return os.path.realpath(common.stdout.strip()), origin_url(where)
|
||||
|
||||
|
||||
def same(one: tuple[str, str], other: tuple[str, str]) -> bool:
|
||||
if one[1] and other[1]:
|
||||
return one[1] == other[1]
|
||||
return one[0] == other[0]
|
||||
|
||||
|
||||
def recorded(row: dict) -> tuple[str, str] | None:
|
||||
common, origin = row.get("common_dir") or "", row.get("origin") or ""
|
||||
if not origin and not os.path.isdir(common):
|
||||
return None
|
||||
return common, origin
|
||||
|
||||
|
||||
def repository(row: dict) -> str:
|
||||
return row.get("origin") or row.get("common_dir") or "-"
|
||||
|
||||
|
||||
def unknown(task: str, row: dict) -> str:
|
||||
return (
|
||||
f"{task}: repository unknown (no origin URL, git dir {row.get('common_dir') or '-'} gone); record it with "
|
||||
f"`hotfix.py update {task.removeprefix(PREFIX)} --status {row['stage']} --sha {row['sha']} --repo <clone>`"
|
||||
)
|
||||
|
||||
|
||||
def hex_id(sha: str) -> str:
|
||||
if not SHA.match(sha):
|
||||
fail(f"--sha {sha!r} is no hex commit id of 7 to 40 characters")
|
||||
return sha
|
||||
|
||||
|
||||
def commit(where: str, sha: str) -> str:
|
||||
found = git(where, "rev-parse", "--verify", "--quiet", f"{hex_id(sha)}^{{commit}}")
|
||||
if found.returncode:
|
||||
fail(f"--sha {sha} is no commit in {where}")
|
||||
return found.stdout.strip()
|
||||
|
||||
|
||||
def default_branch(where: str, given: str | None) -> str:
|
||||
head = git(where, "symbolic-ref", "--quiet", ORIGIN_HEAD)
|
||||
if head.returncode == 0:
|
||||
name = head.stdout.strip().removeprefix("refs/remotes/origin/")
|
||||
if given is not None and given != name:
|
||||
fail(f"origin/HEAD names {name}, --default says {given}")
|
||||
return name
|
||||
if given is None:
|
||||
fail(f"{ORIGIN_HEAD} is missing in {where}; pass --default <branch> or run `git remote set-head origin --auto`")
|
||||
return branch_name(where, given, "--default")
|
||||
|
||||
|
||||
def landed(where: str, sha: str, given: str | None) -> str:
|
||||
hex_id(sha)
|
||||
fetched(where)
|
||||
name = default_branch(where, given)
|
||||
full = commit(where, sha)
|
||||
if not ancestor(where, full, f"refs/remotes/origin/{name}"):
|
||||
fail(f"--sha {sha} is not on origin/{name}; record the commit as it sits there, a squash or rebase merge "
|
||||
"changes it")
|
||||
return full
|
||||
|
||||
|
||||
def branch_name(where: str, name: str, what: str) -> str:
|
||||
if name.startswith("-") or git(where, "check-ref-format", f"refs/heads/{name}").returncode:
|
||||
fail(f"{what} {name!r} is no valid branch name")
|
||||
return name
|
||||
|
||||
|
||||
def children(value: object, subpattern: type) -> list:
|
||||
if isinstance(value, subpattern):
|
||||
return [value]
|
||||
if isinstance(value, (tuple, list)):
|
||||
return [sub for item in value for sub in children(item, subpattern)]
|
||||
return []
|
||||
|
||||
|
||||
def either(tests: list[Callable[[str], bool]]) -> Callable[[str], bool]:
|
||||
return lambda char: any(test(char) for test in tests)
|
||||
|
||||
|
||||
def risk(text: str) -> str | None:
|
||||
parser, c = getattr(re, "_parser", None), getattr(re, "_constants", None)
|
||||
if parser is None or c is None:
|
||||
print("hotfix: this Python hides re's parser; --signature is not checked for runaway backtracking",
|
||||
file=sys.stderr)
|
||||
return None
|
||||
loose = (c.MAX_REPEAT, c.MIN_REPEAT)
|
||||
repeats = (*loose, c.POSSESSIVE_REPEAT)
|
||||
barriers = (c.ATOMIC_GROUP, c.POSSESSIVE_REPEAT)
|
||||
probes = PROBES | set(text)
|
||||
|
||||
def parts(av: object) -> list:
|
||||
return children(av, parser.SubPattern)
|
||||
|
||||
def unbounded(op: object, av: object) -> bool:
|
||||
return op in loose and av[1] == c.MAXREPEAT
|
||||
|
||||
def inside(pattern: object, hit: Callable[[object, object], bool]) -> bool:
|
||||
return any(
|
||||
hit(op, av) or (op not in barriers and any(inside(part, hit) for part in parts(av)))
|
||||
for op, av in pattern.data
|
||||
)
|
||||
|
||||
def overlapping(op: object, av: object) -> bool:
|
||||
if op != c.BRANCH:
|
||||
return False
|
||||
heads = [start(branch) for branch in av[1]]
|
||||
return any(
|
||||
(one_empty and two_empty) or any(one(char) and two(char) for char in probes)
|
||||
for (one, one_empty), (two, two_empty) in itertools.combinations(heads, 2)
|
||||
)
|
||||
|
||||
def start(pattern: object) -> tuple[Callable[[str], bool], bool]:
|
||||
tests = []
|
||||
for op, av in pattern.data:
|
||||
test, empty = first(op, av)
|
||||
tests.append(test)
|
||||
if not empty:
|
||||
return either(tests), False
|
||||
return either(tests), True
|
||||
|
||||
def first(op: object, av: object) -> tuple[Callable[[str], bool], bool]:
|
||||
if op == c.LITERAL:
|
||||
return (lambda char: char == chr(av)), False
|
||||
if op == c.NOT_LITERAL:
|
||||
return (lambda char: char != chr(av)), False
|
||||
if op == c.ANY:
|
||||
return (lambda char: True), False
|
||||
if op == c.IN:
|
||||
return member(av), False
|
||||
if op in (c.AT, c.ASSERT, c.ASSERT_NOT):
|
||||
return (lambda char: False), True
|
||||
if op in repeats:
|
||||
test, empty = start(av[2])
|
||||
return test, empty or av[0] == 0
|
||||
if op in (c.SUBPATTERN, c.ATOMIC_GROUP, c.BRANCH):
|
||||
heads = [start(part) for part in parts(av)]
|
||||
return either([test for test, _ in heads]), any(empty for _, empty in heads)
|
||||
return (lambda char: True), True
|
||||
|
||||
def member(items: list) -> Callable[[str], bool]:
|
||||
negated = bool(items) and items[0][0] == c.NEGATE
|
||||
kept = [item for item in items if item[0] != c.NEGATE]
|
||||
return lambda char: any(holds(op, av, char) for op, av in kept) != negated
|
||||
|
||||
def holds(op: object, av: object, char: str) -> bool:
|
||||
if op == c.LITERAL:
|
||||
return char == chr(av)
|
||||
if op == c.RANGE:
|
||||
return av[0] <= ord(char) <= av[1]
|
||||
if op == c.CATEGORY:
|
||||
name = av.name.removeprefix("CATEGORY_").removeprefix("LOC_").removeprefix("UNI_")
|
||||
klass = CLASSES.get(name.removeprefix("NOT_"))
|
||||
return klass is None or (re.fullmatch(klass, char) is not None) != name.startswith("NOT_")
|
||||
return True
|
||||
|
||||
def scan(pattern: object) -> str | None:
|
||||
for op, av in pattern.data:
|
||||
if unbounded(op, av) and inside(av[2], unbounded):
|
||||
return NESTED
|
||||
if unbounded(op, av) and inside(av[2], overlapping):
|
||||
return OVERLAP
|
||||
for part in parts(av):
|
||||
found = scan(part)
|
||||
if found is not None:
|
||||
return found
|
||||
return None
|
||||
|
||||
return scan(parser.parse(text))
|
||||
|
||||
|
||||
def signature(text: str) -> str:
|
||||
if not text.strip():
|
||||
fail("--signature must not be empty")
|
||||
try:
|
||||
re.compile(text)
|
||||
except re.error as error:
|
||||
fail(f"--signature {text!r}: {error}")
|
||||
found = risk(text)
|
||||
if found is not None:
|
||||
fail(f"--signature {text!r} {found}, which backtracks without end on a near miss; match the stable part "
|
||||
"instead")
|
||||
return text
|
||||
|
||||
|
||||
def searched(patterns: list[str], line: str) -> list[bool]:
|
||||
"""Search the line with every pattern in order in one worker until SEARCH_SECONDS pass.
|
||||
|
||||
Args:
|
||||
patterns: signatures that compile.
|
||||
line: the log line.
|
||||
|
||||
Returns:
|
||||
one verdict per pattern; fewer when the deadline passed, the next pattern still searching.
|
||||
"""
|
||||
payload = json.dumps({"line": line, "patterns": patterns}).encode("ascii")
|
||||
late = False
|
||||
try:
|
||||
with subprocess.Popen(
|
||||
[sys.executable, "-I", "-c", SEARCH], stdin=subprocess.PIPE, stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
) as worker:
|
||||
try:
|
||||
out, err = worker.communicate(payload, timeout=SEARCH_SECONDS)
|
||||
except subprocess.TimeoutExpired:
|
||||
late = True
|
||||
worker.kill()
|
||||
out, err = worker.communicate()
|
||||
except OSError as error:
|
||||
fail(f"cannot start the signature search worker {sys.executable}: {error.strerror or error}")
|
||||
*lines, tail = out.decode("ascii", "replace").split("\n")
|
||||
verdicts = lines[1:]
|
||||
if late and lines[:1] != ["ready"]:
|
||||
fail(f"the signature search worker did not get ready within {SEARCH_SECONDS} s")
|
||||
complete = late or (worker.returncode == 0 and not tail and len(verdicts) == len(patterns))
|
||||
if lines[:1] != ["ready"] or not complete or len(verdicts) > len(patterns) or set(verdicts) - {"0", "1"}:
|
||||
reason = err.decode("utf-8", "replace").strip().splitlines()[-1:] or ["no message"]
|
||||
fail(f"the signature search worker exited {worker.returncode}: {reason[0]}")
|
||||
return [verdict == "1" for verdict in verdicts]
|
||||
|
||||
|
||||
def make_targets(values: list[str] | None) -> list[str]:
|
||||
for value in values or []:
|
||||
if not TARGET.match(value):
|
||||
fail(f"--target {value!r} is no make target name")
|
||||
return list(dict.fromkeys(values or []))
|
||||
|
||||
|
||||
def turns_green(greens: list[str], wanted: set[str]) -> bool:
|
||||
if not greens or wanted & set(greens):
|
||||
return True
|
||||
return UMBRELLA in wanted and any(green.startswith(f"{UMBRELLA}-") for green in greens)
|
||||
|
||||
|
||||
def number(value: int | None) -> int | None:
|
||||
if value is not None and value < 1:
|
||||
fail("--pr must be 1 or higher")
|
||||
return value
|
||||
|
||||
|
||||
def hotfixes(rasci: ModuleType) -> list[tuple[str, dict]]:
|
||||
return [(t, row) for t, row in sorted(rasci.load().items()) if t.startswith(PREFIX) and "stage" in row]
|
||||
|
||||
|
||||
def of_repository(rasci: ModuleType, repo: str) -> tuple[list[tuple[str, dict]], list[str]]:
|
||||
here = identity(repo)
|
||||
rows, unknowns = [], []
|
||||
for task, row in hotfixes(rasci):
|
||||
theirs = recorded(row)
|
||||
if theirs is None:
|
||||
unknowns.append(unknown(task, row))
|
||||
elif same(theirs, here):
|
||||
rows.append((task, row))
|
||||
return rows, unknowns
|
||||
|
||||
|
||||
def for_targets(rows: list[tuple[str, dict]], targets: list[str] | None) -> list[tuple[str, dict]]:
|
||||
if targets is None:
|
||||
return rows
|
||||
wanted = set(make_targets(targets))
|
||||
return [(task, row) for task, row in rows if turns_green(row.get("targets") or [], wanted)]
|
||||
|
||||
|
||||
def missing(repo: str, rows: list[tuple[str, dict]], problems: list[str]) -> tuple[list, list, str]:
|
||||
absent = [(t, row) for t, row in rows if row["stage"] == "merged" and not ancestor(repo, row["sha"], "HEAD")]
|
||||
if not absent:
|
||||
return [], [], ""
|
||||
head = git(repo, "symbolic-ref", "--quiet", ORIGIN_HEAD)
|
||||
if head.returncode:
|
||||
problems.append(f"{ORIGIN_HEAD} is missing in {repo}, so {', '.join(t for t, _ in absent)} stay unchecked; "
|
||||
"run `git remote set-head origin --auto`")
|
||||
return [], [], ""
|
||||
ref = head.stdout.strip()
|
||||
on = {t for t, row in absent if ancestor(repo, row["sha"], ref)}
|
||||
if len(on) < len(absent):
|
||||
result = fetch(repo)
|
||||
if result.returncode:
|
||||
unchecked = ", ".join(t for t, _ in absent if t not in on)
|
||||
problems.append(f"git fetch origin in {repo} failed, so {unchecked} stay unchecked: {result.stderr.strip()}")
|
||||
return [(t, row) for t, row in absent if t in on], [], ""
|
||||
on = {t for t, row in absent if ancestor(repo, row["sha"], ref)}
|
||||
shown = ref.removeprefix("refs/remotes/")
|
||||
return [(t, row) for t, row in absent if t in on], [(t, row) for t, row in absent if t not in on], shown
|
||||
|
||||
|
||||
def pr_cell(row: dict) -> str:
|
||||
return f"#{row['pr']}" if row.get("pr") else "-"
|
||||
|
||||
|
||||
def cmd_add(rasci: ModuleType, args: argparse.Namespace) -> int:
|
||||
task = task_of(rasci, args.slug)
|
||||
if task in rasci.load():
|
||||
fail(f"{task} is already in the register; move it on with update")
|
||||
repo = clone(args.repo)
|
||||
common, origin = identity(repo)
|
||||
entry = {
|
||||
"common_dir": common,
|
||||
"origin": origin,
|
||||
"branch": branch_name(repo, args.branch, "--branch"),
|
||||
"signature": signature(args.signature),
|
||||
"targets": make_targets(args.target),
|
||||
"stage": args.status,
|
||||
}
|
||||
pr = number(args.pr)
|
||||
if pr is not None:
|
||||
entry["pr"] = pr
|
||||
entry["sha"] = landed(repo, args.sha, args.default) if args.status == "merged" else commit(repo, args.sha)
|
||||
council = module("council", rasci.COUNCIL)
|
||||
sid, _ = council.caller(args.session)
|
||||
entry["roles"] = {council.live_sessions().get(sid) or sid: "RA"}
|
||||
rasci.apply_status(entry, REGISTER_STATUS[args.status])
|
||||
with rasci.database() as tasks:
|
||||
if task in tasks:
|
||||
fail(f"{task} is already in the register; move it on with update")
|
||||
tasks[task] = entry
|
||||
print(f"added {task}")
|
||||
return 0
|
||||
|
||||
|
||||
def located(task: str, entry: dict, given: str | None) -> tuple[str, dict]:
|
||||
if given is None:
|
||||
common = entry.get("common_dir") or ""
|
||||
if not os.path.isdir(common):
|
||||
fail(f"{task}: its git dir {common or '-'} is gone; pass --repo <a clone of {repository(entry)}>")
|
||||
return common, {}
|
||||
repo = clone(given)
|
||||
here = identity(repo)
|
||||
theirs = recorded(entry)
|
||||
if theirs is not None and not same(theirs, here):
|
||||
fail(f"--repo {given} is another repository than {task}'s {repository(entry)}")
|
||||
return repo, {"common_dir": here[0], "origin": here[1]}
|
||||
|
||||
|
||||
def cmd_update(rasci: ModuleType, args: argparse.Namespace) -> int:
|
||||
task = task_of(rasci, args.slug)
|
||||
pr = number(args.pr)
|
||||
if args.status == "merged" and args.sha is None:
|
||||
fail("--status merged needs --sha, the commit as it sits on the default branch")
|
||||
entry = rasci.load().get(task)
|
||||
if entry is None or "stage" not in entry:
|
||||
fail(f"{task}: no hotfix in the register; record it with add")
|
||||
fields: dict = {}
|
||||
if args.repo is not None or args.sha is not None:
|
||||
where, fields = located(task, entry, args.repo)
|
||||
if args.sha is not None:
|
||||
fields["sha"] = landed(where, args.sha, args.default) if args.status == "merged" else commit(where, args.sha)
|
||||
if pr is not None:
|
||||
fields["pr"] = pr
|
||||
with rasci.database() as tasks:
|
||||
entry = tasks.get(task)
|
||||
if entry is None or "stage" not in entry:
|
||||
fail(f"{task}: no hotfix in the register; record it with add")
|
||||
entry.update(fields)
|
||||
entry["stage"] = args.status
|
||||
rasci.apply_status(entry, REGISTER_STATUS[args.status])
|
||||
print(f"updated {task}")
|
||||
return 0
|
||||
|
||||
|
||||
def cmd_pending(rasci: ModuleType, args: argparse.Namespace) -> int:
|
||||
repo = worktree(args.repo)
|
||||
rows, unknowns = of_repository(rasci, repo)
|
||||
rows = for_targets(rows, args.target)
|
||||
problems: list[str] = []
|
||||
on, off, shown = missing(repo, rows, problems)
|
||||
verdicts = {t: "missing (merge main)" for t, _ in on} | {t: f"not on {shown}" for t, _ in off}
|
||||
opened = 0
|
||||
for task, row in rows:
|
||||
if row["stage"] != "merged":
|
||||
verdict = "open, not merged"
|
||||
opened += 1
|
||||
elif task in verdicts:
|
||||
verdict = verdicts[task]
|
||||
else:
|
||||
continue
|
||||
print("\t".join((verdict, task, row["stage"], row["branch"], row["sha"], pr_cell(row))))
|
||||
report([*unknowns, *problems])
|
||||
print(f"hotfix: {len(on)} missing, {opened} open")
|
||||
if problems:
|
||||
return 2
|
||||
return 1 if on else 0
|
||||
|
||||
|
||||
def cmd_match(rasci: ModuleType, args: argparse.Namespace) -> int:
|
||||
here = identity(worktree(args.repo)) if args.repo is not None else None
|
||||
rows = []
|
||||
for task, row in hotfixes(rasci):
|
||||
try:
|
||||
re.compile(row["signature"])
|
||||
except re.error as error:
|
||||
print(f"hotfix: {task}: signature does not compile ({error}), skipped", file=sys.stderr)
|
||||
continue
|
||||
rows.append((task, row))
|
||||
hits = []
|
||||
while rows:
|
||||
verdicts = searched([row["signature"] for _, row in rows], args.line)
|
||||
hits += [entry for entry, hit in zip(rows[:len(verdicts)], verdicts, strict=True) if hit]
|
||||
if len(verdicts) == len(rows):
|
||||
break
|
||||
task, row = rows[len(verdicts)]
|
||||
print(f"hotfix: {task}: signature {row['signature']!r} still searched this line at the {SEARCH_SECONDS} s "
|
||||
"deadline, skipped; its A must narrow it", file=sys.stderr)
|
||||
rows = rows[len(verdicts) + 1:]
|
||||
found = 0
|
||||
for task, row in hits:
|
||||
cells = [task, row["stage"], row["branch"], row["sha"], pr_cell(row), repository(row)]
|
||||
theirs = recorded(row)
|
||||
if here is not None and theirs is None:
|
||||
cells.insert(0, "unknown")
|
||||
elif here is not None and not same(theirs, here):
|
||||
cells.insert(0, "foreign")
|
||||
else:
|
||||
found += 1
|
||||
print("\t".join(cells))
|
||||
if not found:
|
||||
print("hotfix: no signature matches" if here is None else "hotfix: no signature of this repository matches")
|
||||
return 0 if found else 1
|
||||
|
||||
|
||||
def cmd_void(rasci: ModuleType, args: argparse.Namespace) -> int:
|
||||
repo = worktree(args.repo)
|
||||
rows, unknowns = of_repository(rasci, repo)
|
||||
rows = for_targets(rows, args.target)
|
||||
problems: list[str] = []
|
||||
on, off, shown = missing(repo, rows, problems)
|
||||
for task, row in off:
|
||||
print(f"hotfix: {task}: {row['sha']} is not on {shown} (a squash or rebase merge?), so it voids nothing; "
|
||||
"its A must record the commit as it sits there", file=sys.stderr)
|
||||
for task, row in on:
|
||||
print("\t".join(("void", task, row["sha"], ",".join(row.get("targets") or []) or "all targets")))
|
||||
report([*unknowns, *problems])
|
||||
if on:
|
||||
return 3
|
||||
return 2 if problems else 0
|
||||
|
||||
|
||||
@contextlib.contextmanager
|
||||
def locked(repo: str, command: list[str]) -> Iterator[None]:
|
||||
"""Hold gate.py's sync lock and worktree lock of the worktree.
|
||||
|
||||
Args:
|
||||
repo: the worktree.
|
||||
command: sync's command line, recorded in the sync lock.
|
||||
"""
|
||||
if not GATE.is_file():
|
||||
yield
|
||||
return
|
||||
gate = module("gate", GATE)
|
||||
git_dir = Path(git_out(repo, "rev-parse", "--absolute-git-dir").strip())
|
||||
with (
|
||||
open(git_dir / gate.SYNC_LOCK, "a", encoding="utf-8") as sync,
|
||||
open(git_dir / gate.RUN_LOCK, "a", encoding="utf-8") as run,
|
||||
):
|
||||
fcntl.flock(sync, fcntl.LOCK_EX)
|
||||
gate.claim(sync, command)
|
||||
try:
|
||||
fcntl.flock(run, fcntl.LOCK_EX | fcntl.LOCK_NB)
|
||||
except BlockingIOError:
|
||||
fail(f"a gate.py run or hold holds {repo} ({gate.holder(git_dir / gate.RUN_LOCK)}); sync once it ends")
|
||||
yield
|
||||
|
||||
|
||||
def watched(repo: str) -> None:
|
||||
if WATCH.is_file():
|
||||
watch = module("watch", WATCH)
|
||||
config = watch.load_config()
|
||||
if config is not None and watch.busy(repo, config["busy_processes"]):
|
||||
fail(f"a busy process (watch.py busy_processes) runs in {repo}, a gate or deploy; sync once it ends")
|
||||
|
||||
|
||||
def chain(repo: str, tasks: dict[str, dict], task: str, default: str, ended: tuple[str, ...]) -> list[str]:
|
||||
here = identity(repo)
|
||||
places: dict[str, tuple[str, str] | None] = {}
|
||||
found: list[str] = []
|
||||
base = tasks[task].get("base")
|
||||
while base and base != default:
|
||||
if base in found:
|
||||
fail(f"base chain of {task} loops: {' <- '.join([*found, base])}")
|
||||
found.append(branch_name(repo, base, "base"))
|
||||
upstream = {}
|
||||
for name, row in sorted(tasks.items()):
|
||||
if row.get("branch") != base or not row.get("base") or row.get("status") in ended:
|
||||
continue
|
||||
workdir = row.get("workdir") or ""
|
||||
if not workdir:
|
||||
if not (resolves(repo, f"refs/heads/{base}") or resolves(repo, f"refs/remotes/origin/{base}")):
|
||||
fail(f"{name} records no workdir and its branch {base} resolves neither as {base} nor as "
|
||||
f"origin/{base} in {repo}; record it with `rasci.py set {name} --workdir <dir>`")
|
||||
upstream[name] = row["base"]
|
||||
continue
|
||||
if workdir not in places:
|
||||
places[workdir] = identity(workdir)
|
||||
if places[workdir] is None:
|
||||
print(f"hotfix: {name}: workdir {workdir} is no git repository, its base is not followed",
|
||||
file=sys.stderr)
|
||||
elif same(places[workdir], here):
|
||||
upstream[name] = row["base"]
|
||||
if len(set(upstream.values())) > 1:
|
||||
named = ", ".join(f"{t}={b}" for t, b in sorted(upstream.items()))
|
||||
fail(f"branch {base} has several bases in the register: {named}")
|
||||
base = next(iter(upstream.values()), None)
|
||||
return found
|
||||
|
||||
|
||||
def base_ref(repo: str, base: str) -> tuple[str, str]:
|
||||
local, remote = f"refs/heads/{base}", f"refs/remotes/origin/{base}"
|
||||
if not resolves(repo, remote):
|
||||
if not resolves(repo, local):
|
||||
fail(f"{base} does not resolve in {repo}")
|
||||
return base, local
|
||||
if not resolves(repo, local) or ancestor(repo, local, remote):
|
||||
return f"origin/{base}", remote
|
||||
if ancestor(repo, remote, local):
|
||||
return base, local
|
||||
fail(f"{base} and origin/{base} have diverged in {repo}; merge one into the other first")
|
||||
|
||||
|
||||
def default_ref(repo: str, given: str | None) -> tuple[str, tuple[str, str]]:
|
||||
if git(repo, "config", "--get", "remote.origin.url").stdout.strip():
|
||||
default = default_branch(repo, given)
|
||||
fetched(repo)
|
||||
return default, (f"origin/{default}", f"refs/remotes/origin/{default}")
|
||||
if given is None:
|
||||
fail(f"{repo} has no origin remote; pass --default <branch>, its local default branch")
|
||||
default = branch_name(repo, given, "--default")
|
||||
return default, (default, f"refs/heads/{default}")
|
||||
|
||||
|
||||
def cmd_sync(rasci: ModuleType, args: argparse.Namespace) -> int:
|
||||
repo = worktree(args.repo)
|
||||
tasks = rasci.load()
|
||||
if args.task not in tasks:
|
||||
fail(f"{args.task}: not in the register")
|
||||
branch = tasks[args.task].get("branch")
|
||||
if not branch:
|
||||
fail(f"{args.task} records no branch; set it with `rasci.py set {args.task} --branch <branch>`")
|
||||
head = git(repo, "symbolic-ref", "--quiet", "HEAD").stdout.strip()
|
||||
if head != f"refs/heads/{branch}":
|
||||
fail(f"{repo} has {head.removeprefix('refs/heads/') or 'a detached HEAD'} checked out, "
|
||||
f"{args.task} works on {branch}")
|
||||
default, first = default_ref(repo, args.default)
|
||||
with locked(repo, ["hotfix.py", "sync", "--repo", repo, "--task", args.task]):
|
||||
watched(repo)
|
||||
dirty = [line[3:] for line in git_out(repo, "status", "--porcelain=v1", "--untracked-files=no").splitlines()]
|
||||
if dirty:
|
||||
fail(f"tracked files are dirty, commit them first: {', '.join(dirty)}")
|
||||
if not resolves(repo, first[1]):
|
||||
fail(f"{first[0]} does not resolve in {repo}")
|
||||
bases = chain(repo, tasks, args.task, default, rasci.ENDED)
|
||||
for shown, ref in [first, *(base_ref(repo, base) for base in bases)]:
|
||||
if ancestor(repo, ref, "HEAD"):
|
||||
print(f"contained\t{shown}")
|
||||
continue
|
||||
result = git(repo, "merge", "--no-verify", "--no-edit", ref)
|
||||
if result.returncode == 0:
|
||||
print(f"merged\t{shown}\t{git_out(repo, 'rev-parse', 'HEAD').strip()}")
|
||||
continue
|
||||
files = git(repo, "diff", "--name-only", "--diff-filter=U").stdout.splitlines()
|
||||
if git(repo, "rev-parse", "--verify", "--quiet", "MERGE_HEAD").returncode == 0:
|
||||
git_out(repo, "merge", "--abort")
|
||||
if not files:
|
||||
fail(f"git merge {shown} failed and was aborted: {(result.stderr or result.stdout).strip()}")
|
||||
print(f"conflict\t{shown}\t{','.join(files)}")
|
||||
print(f"hotfix: merging {shown} conflicts in {', '.join(files)}; merge aborted", file=sys.stderr)
|
||||
return 4
|
||||
return 0
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
|
||||
sub = parser.add_subparsers(dest="cmd", required=True)
|
||||
p = sub.add_parser("add")
|
||||
p.add_argument("slug")
|
||||
for flag in ("--repo", "--branch", "--sha", "--signature"):
|
||||
p.add_argument(flag, required=True)
|
||||
p.add_argument("--status", choices=STAGES, required=True)
|
||||
p.add_argument("--target", action="append")
|
||||
p.add_argument("--pr", type=int)
|
||||
p.add_argument("--default")
|
||||
p.add_argument("--session")
|
||||
p.set_defaults(func=cmd_add)
|
||||
p = sub.add_parser("update")
|
||||
p.add_argument("slug")
|
||||
p.add_argument("--status", choices=STAGES, required=True)
|
||||
p.add_argument("--sha")
|
||||
p.add_argument("--pr", type=int)
|
||||
p.add_argument("--repo")
|
||||
p.add_argument("--default")
|
||||
p.set_defaults(func=cmd_update)
|
||||
p = sub.add_parser("pending")
|
||||
p.add_argument("--repo", required=True)
|
||||
p.add_argument("--target", action="append")
|
||||
p.set_defaults(func=cmd_pending)
|
||||
p = sub.add_parser("match")
|
||||
p.add_argument("--line", required=True)
|
||||
p.add_argument("--repo")
|
||||
p.set_defaults(func=cmd_match)
|
||||
p = sub.add_parser("void")
|
||||
p.add_argument("--repo", required=True)
|
||||
p.add_argument("--target", action="append", required=True)
|
||||
p.set_defaults(func=cmd_void)
|
||||
p = sub.add_parser("sync")
|
||||
p.add_argument("--repo", required=True)
|
||||
p.add_argument("--task", required=True)
|
||||
p.add_argument("--default")
|
||||
p.set_defaults(func=cmd_sync)
|
||||
args = parser.parse_args()
|
||||
return args.func(module("rasci", RASCI), args)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -8,5 +8,8 @@ description: >
|
||||
|
||||
Run the `kvb-org-organize` skill's **Peer protocol**, its `lineage.md`
|
||||
included, for the failure or fix at hand. They hold the whole procedure,
|
||||
including the entries in the `kvb-org-rasci` register: ask by `SendMessage`,
|
||||
report by `news.py post`. This skill only keeps its command working.
|
||||
including the entries in the `kvb-org-rasci` register: look first, `hotfix.py
|
||||
match` of the `kvb-git-hotfix` skill included; ask by `SendMessage`; report
|
||||
by `news.py post`. A hotfix is recorded with `hotfix.py add` and `update`,
|
||||
never reported by message or news post. This skill only keeps its command
|
||||
working.
|
||||
@@ -7,7 +7,7 @@ description: >
|
||||
---
|
||||
|
||||
The first claim owns a piece of work; a later agent hands its findings to the
|
||||
owner and never builds a second version. Offices are recallable mandates
|
||||
owner, never building a second version. Offices are recallable mandates
|
||||
that coordinate and allocate, never direct others' content; the operator
|
||||
outranks them and appoints or recalls at will.
|
||||
|
||||
@@ -18,15 +18,15 @@ outranks them and appoints or recalls at will.
|
||||
|
||||
1. Name the office: `work/<slug>` (a piece of work), `cluster/<slug>` (a
|
||||
strand across lines), `<dimension>/<line>` (a line) or `chair`, the slug
|
||||
concrete (issue number, failing line, log line). The first agent elected
|
||||
concrete (issue number, failing log line). The first agent elected
|
||||
while `chair` is vacant also claims it. Read `offices.md` next to this
|
||||
file before claiming a cluster, a line or the chair.
|
||||
2. `council.py list`; the same work claimed under another slug counts.
|
||||
3. `council.py claim <office> --work "<what, evidence, branch or worktree>"`.
|
||||
Exit 0: yours; 3: a live holder (printed); 4: the holder's session is dead.
|
||||
Exit 0: yours; 3: a live holder (printed); 4: a dead holder.
|
||||
4. Elected: complete the task the claim entered in the `kvb-org-rasci`
|
||||
register (objective, description, priority, roles); the listed claim is
|
||||
the announcement.
|
||||
register (objective, description, priority, roles); the listed claim
|
||||
announces it.
|
||||
5. Not elected: post your findings to the holder via the news and stop.
|
||||
|
||||
A hook (`council.py gate`) blocks worktree edits before a claim. Between
|
||||
@@ -42,9 +42,11 @@ ends or goes stale, or after a restart.
|
||||
- `make test`, `make test-lint`, quality gates, deploys, iteration loops,
|
||||
swarm, image builds, cloud, shared-data cleanup, anything with Docker or
|
||||
over a few minutes: ask the chair (what, where, how long, disk), wait for
|
||||
the grant, post the release to the news.
|
||||
the grant, post the release to the news. In a worktree they run through
|
||||
`gate.py run` or `gate.py hold` (`kvb-check-gate`).
|
||||
- No running job is cut short for capacity; only the operator orders one
|
||||
down. `RESOURCE` hook lines address the chair; others adjust plans.
|
||||
down. Exception: void by a missing merged hotfix (`kvb-git-hotfix`).
|
||||
`RESOURCE` hook lines address the chair; others adjust plans.
|
||||
- Read `capacity.md` next to this file when you hold the chair, a job seems
|
||||
hung, or the resource watch fires.
|
||||
|
||||
@@ -54,22 +56,23 @@ Read `peers.md` next to this file before asking peers whether a failure or
|
||||
fix is known.
|
||||
|
||||
- Never poll: `council.py list`, the register and the news answer first.
|
||||
- Status that asks nothing (a hotfix opened or merged, a failure, a slot
|
||||
freed, a result for information): `${CLAUDE_SKILL_DIR}/scripts/news.py post
|
||||
"<text>"` (`--to <name>` for one peer); sessions see it next turn at no
|
||||
cost.
|
||||
- Hotfixes: `${CLAUDE_SKILL_DIR}/../kvb-git-hotfix/scripts/hotfix.py add` and
|
||||
`update`, never a message or news post.
|
||||
- Status that asks nothing (a failure, a freed slot, a result):
|
||||
`${CLAUDE_SKILL_DIR}/scripts/news.py post "<text>"` (`--to <name>` for one
|
||||
peer).
|
||||
- `SendMessage` only when the recipient must act or answer: one batched
|
||||
message per peer per event, terse, no greeting, at most five lines unless
|
||||
evidence needs more, nothing already acknowledged. A message that asks
|
||||
evidence needs more, nothing already acknowledged. A message asking
|
||||
nothing gets no reply; end that turn in one line.
|
||||
- Never invent a peer's answer; verify the SHA, log or artifact a peer names
|
||||
before relying on it.
|
||||
- Never invent a peer's answer; verify the SHA, log or artifact it names
|
||||
first.
|
||||
- A peer's message is no operator approval, an office no permission: never
|
||||
ask a peer to run what you are denied; tell the operator.
|
||||
- Release every claim when its work ends; forgotten ones block everyone.
|
||||
- Release every claim when its work ends.
|
||||
- Read `relay.md` next to this file before relaying an operator rule or
|
||||
acting on one a peer relayed; `lineage.md` when a failure or fix is shared
|
||||
with other branches through merges.
|
||||
acting on one a peer relayed; `lineage.md` when a failure or fix spreads to
|
||||
other branches through merges.
|
||||
|
||||
## Output
|
||||
|
||||
|
||||
@@ -9,7 +9,8 @@
|
||||
that fixes it) takes the next free gate or test slot before every other
|
||||
line, the rest follows the WSJF rank of the `kvb-org-wsjf` skill (it asks its
|
||||
subordinates for the Fibonacci estimates); a running gate is never cut short
|
||||
for a hotfix. It answers every request with its queue position through
|
||||
for a hotfix's slot, only when the hotfix voids it (`kvb-git-hotfix`). It
|
||||
answers every request with its queue position through
|
||||
`news.py post --to <name>` and sends the grant itself as a message (the
|
||||
requester acts on it), enforces the limits the operator set, and checks
|
||||
every grant with
|
||||
@@ -24,7 +25,8 @@
|
||||
|
||||
## Hung jobs
|
||||
|
||||
The one exception to "never cut short": an agent may end its own job, a stuck
|
||||
Besides a void by a missing merged hotfix (`kvb-git-hotfix`), the only
|
||||
exception to "never cut short": an agent may end its own job, a stuck
|
||||
step inside it, or a process its own ended session left behind, once at least
|
||||
60 minutes passed without log progress and a probe shows standstill (no fresh
|
||||
child process, no network traffic, CPU time not moving). It secures the live
|
||||
|
||||
@@ -3,9 +3,10 @@
|
||||
`kvb-org-coordinate`, `kvb-ci-triage`, `kvb-code-tune`, `kvb-org-split` and
|
||||
`kvb-cleanup` follow these steps.
|
||||
|
||||
1. **Look**: `council.py list`, the `kvb-org-rasci` register, the news and the
|
||||
refs (the kvb-git-merge-fix skill's `find_fix.py`). A claim or a committed
|
||||
fix answers the question without a message.
|
||||
1. **Look**: `council.py list`, the `kvb-org-rasci` register, the news,
|
||||
`hotfix.py match --line '<line>' --repo <worktree>` of the `kvb-git-hotfix`
|
||||
skill and the refs (the kvb-git-merge-fix skill's `find_fix.py`). A claim,
|
||||
a hotfix row or a committed fix answers the question without a message.
|
||||
2. **Claim** the office (Election in `SKILL.md`).
|
||||
3. **Ask** (in Claude Code: `ListAgents`, then `SendMessage`) only the peers
|
||||
whose lines the work touches, the lineage owners (`lineage.md`) and, for
|
||||
@@ -21,7 +22,8 @@
|
||||
neither a "no" nor a competing claim: after 5 minutes, proceed.
|
||||
5. **Report** to the same peers through `news.py post --to <name>`: commit,
|
||||
branch, what changed, the evidence it works, and the merge it reaches them
|
||||
by; an abandoned or failed attempt too. In the register, enter the peers
|
||||
told as I and mark every key result the evidence proves as met. Then
|
||||
release the claim: the task ends `done`, or `stopped` while a key result is
|
||||
open.
|
||||
by; an abandoned or failed attempt too. A hotfix is no report: it is
|
||||
recorded with `hotfix.py add` and `update`, never by message or news post.
|
||||
In the register, enter the peers told as I and mark every key result the
|
||||
evidence proves as met. Then release the claim: the task ends `done`, or
|
||||
`stopped` while a key result is open.
|
||||
@@ -13,7 +13,7 @@ itself (holder as R and A, `running`, start, workdir, branch); its release
|
||||
sets `done` and the end, or `stopped` while a key result is still open.
|
||||
|
||||
```bash
|
||||
rasci.py set <task> --title <t> --objective <o> --description <d> --prio <n> --status queued|running|blocked|done|stopped --worker "<agent | subagent <type> (<model>)>" --workdir <dir> --branch <b> --artifact <kind>:<ref> --tool <script or command> [--start <iso>] [--end <iso>] [--drop-artifact <a>] [--drop-tool <t>]
|
||||
rasci.py set <task> --title <t> --objective <o> --description <d> --prio <n> --status queued|running|blocked|done|stopped --worker "<agent | subagent <type> (<model>)>" --workdir <dir> --branch <b> --base <upstream branch> --artifact <kind>:<ref> --tool <script or command> [--start <iso>] [--end <iso>] [--drop-artifact <a>] [--drop-tool <t>]
|
||||
rasci.py kr <task> add "<measurable result>" | met <n> | open <n> | rm <n>
|
||||
rasci.py role <task> "<party>" <letters>
|
||||
rasci.py get <task>
|
||||
@@ -68,5 +68,6 @@ published). `done` is refused while one is open; a task ended short of them is
|
||||
`get` instead of asking a peer what it works on. `show --wide` adds OKR,
|
||||
Artifacts, Start, End and every RASCI letter.
|
||||
- When invoked, print `show --wide` unchanged; its `gaps:` line names tasks
|
||||
without exactly one A, without R, or open without a key result. `get`
|
||||
prints one task in full, key results numbered.
|
||||
without exactly one A, without R, or open without a key result. `hotfix/`
|
||||
and `operator/` rows are records, not work tasks: `gaps:`, `wsjf` and
|
||||
`artifacts` skip them. `get` prints one task in full, key results numbered.
|
||||
@@ -39,13 +39,14 @@ STATUSES = ("queued", "running", "blocked", "done", "stopped")
|
||||
ENDED = ("done", "stopped")
|
||||
FIELDS = (
|
||||
"title", "objective", "description", "prio", "status", "worker", "start", "end", "workdir",
|
||||
"branch",
|
||||
"branch", "base",
|
||||
)
|
||||
TEXT_FIELDS = ("title", "objective", "description", "worker", "workdir", "branch")
|
||||
TEXT_FIELDS = ("title", "objective", "description", "worker", "workdir", "branch", "base")
|
||||
LIST_FIELDS = {"tools": "tool"}
|
||||
ARTIFACT = re.compile(r"^[a-z]+:\S")
|
||||
FIBONACCI = (1, 2, 3, 5, 8, 13, 21, 34, 55, 89)
|
||||
OPERATOR = "Operator"
|
||||
RECORDS = ("hotfix/", "operator/")
|
||||
|
||||
|
||||
def fail(message: str) -> None:
|
||||
@@ -256,6 +257,8 @@ def okr(row: dict) -> str:
|
||||
|
||||
|
||||
def gaps(task: str, row: dict) -> list[str]:
|
||||
if task.startswith(RECORDS):
|
||||
return []
|
||||
found = []
|
||||
roles = row["roles"]
|
||||
accountable = sum("A" in letters for letters in roles.values())
|
||||
@@ -388,7 +391,7 @@ def cmd_estimate(args: argparse.Namespace) -> int:
|
||||
|
||||
|
||||
def cmd_wsjf(args: argparse.Namespace) -> int:
|
||||
rows = merged(args.all)
|
||||
rows = {task: row for task, row in merged(args.all).items() if not task.startswith(RECORDS)}
|
||||
if not rows:
|
||||
print("no tasks")
|
||||
return 0
|
||||
@@ -414,7 +417,7 @@ def cmd_artifacts(args: argparse.Namespace) -> int:
|
||||
rows = merged(True)
|
||||
for task in sorted(rows):
|
||||
row = rows[task]
|
||||
if args.task and task != args.task:
|
||||
if task.startswith(RECORDS) or (args.task and task != args.task):
|
||||
continue
|
||||
if args.ended and row["status"] not in (*ENDED, "stale"):
|
||||
continue
|
||||
|
||||
@@ -39,14 +39,16 @@ in a sidetree until it does, or leaves as a proposal.
|
||||
never into a message.
|
||||
2. **Sidetree.** Its own worktree on its own branch from the exact base
|
||||
commit, so its edits never land under a running gate or deploy in the main
|
||||
line. Use the host's worktree isolation when it has one.
|
||||
line. Use the host's worktree isolation if any. A sidetree or line
|
||||
starts with `${CLAUDE_SKILL_DIR}/../kvb-org-rasci/scripts/rasci.py set
|
||||
<task> --branch <own> --base <upstream> --workdir <dir>`.
|
||||
3. **Evidence.** Targeted tests, a test for the new behaviour, and a negative
|
||||
control: the test fails without the change. Numbers come from the same
|
||||
source before and after.
|
||||
4. **Self-assessment.** The agent reports fix confidence and break risk with
|
||||
their residuals and commits on its branch. Its parent marks proven key
|
||||
results met in the `kvb-org-rasci` register and records
|
||||
`--branch` and `--artifact worktree:<path>`.
|
||||
`--artifact worktree:<path>`.
|
||||
5. **Independent review.** Before any merge, run the `kvb-think-dialectic`
|
||||
skill with skeptics who did not write the change, briefed the same way
|
||||
(exact diff and claim, capped answer). A hole returns the change to stage 3.
|
||||
@@ -58,9 +60,8 @@ in a sidetree until it does, or leaves as a proposal.
|
||||
|
||||
## When the bar is out of reach
|
||||
|
||||
When the deciding evidence (a live deploy, a production-like dataset, an
|
||||
upstream answer) is not available to the agent or the change is inherently
|
||||
riskier, read `proposal.md` next to this file.
|
||||
When the deciding evidence is not available to the agent or the change is
|
||||
inherently riskier, read `proposal.md` next to this file.
|
||||
|
||||
## Rules
|
||||
|
||||
|
||||
@@ -1,6 +1,8 @@
|
||||
# Proposal
|
||||
|
||||
When the bar is out of reach, the agent stops trying and submits a proposal:
|
||||
When the bar is out of reach (the deciding evidence, such as a live deploy, a
|
||||
production-like dataset or an upstream answer, is not available, or the change
|
||||
is inherently riskier), the agent stops trying and submits a proposal:
|
||||
|
||||
- what the change does and why it is worth having;
|
||||
- the confidence and break risk it reached, with their residuals;
|
||||
|
||||
Reference in new issue
Block a user