feat(kvb-check-gate): prove 90/5 by every make test* target on the exact head
The operator's rule (2026-11-03): nothing is declared 90/5 before every make test* target of the project ran green on the exact head. kvb-check-gate holds it as the single source and gate.py backs it: list parses the Makefile make would read and its includes statically (never running make, whose $(shell ...) would execute), status reports green, red or missing per target for HEAD and says gate: green only when all are green on a clean tracked tree, and run executes known targets one by one as make -C with output written straight to a log (no pipe hiding the exit code), marking a target red on a non-zero exit, a make: *** or make[N]: *** line, or a project --fail-regex such as infinito-nexus' FAILED TARGETS. Results live per sha in the git common dir under a lock, shared by linked worktrees; a new commit or a target that edits tracked files invalidates them. kvb-check-confidence and kvb-git-pr-label point to gate.py status instead of restating the rule; the README places the skill. test_gate covers parsing with includes, green, red and fail-pattern runs, stale results after a commit, dirty trees, worktrees and usage errors. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
3c2f6c44b5
commit
e581025736
6 files changed
+644
-6
No files matched your search
@@ -62,6 +62,7 @@ flowchart LR
|
||||
check --> check_principles[kvb-check-principles]
|
||||
check --> check_confidence[kvb-check-confidence]
|
||||
check --> check_break_risk[kvb-check-break-risk]
|
||||
check --> check_gate[kvb-check-gate]
|
||||
org --> org_organize[kvb-org-organize]
|
||||
org --> org_split[kvb-org-split]
|
||||
org --> org_coordinate[kvb-org-coordinate]
|
||||
@@ -155,6 +156,7 @@ flowchart LR
|
||||
sceptic[kvb-check-principles]
|
||||
confidence[kvb-check-confidence]
|
||||
break_risk[kvb-check-break-risk]
|
||||
gate[kvb-check-gate]
|
||||
end
|
||||
subgraph cat_org [org]
|
||||
democracy[kvb-org-organize]
|
||||
@@ -227,6 +229,9 @@ flowchart LR
|
||||
commit_no_verify --> advocate
|
||||
advocate --> dialectic
|
||||
confidence --> break_risk
|
||||
confidence --> gate
|
||||
pr_label --> gate
|
||||
gate --> democracy
|
||||
confidence -. objective .-> autotune
|
||||
|
||||
coordinate --> democracy
|
||||
|
||||
@@ -32,10 +32,8 @@ Calibration rules:
|
||||
|
||||
A threshold the operator named ("95% confidence, break risk ≤5%") is a gate on
|
||||
the work for both numbers, handled as the `kvb-check-break-risk` skill's last
|
||||
section says. Fix confidence of 90 % or more with break risk under 5 % (90/5)
|
||||
needs the project's full quality gate green on the exact head (in
|
||||
infinito-nexus `make quality-high`); before that, the numbers are
|
||||
provisional and the missing gate is the residual.
|
||||
section says. 90/5 also needs `gate: green` from `gate.py status` of the
|
||||
`kvb-check-gate` skill, which states the rule and its residuals.
|
||||
|
||||
Objective: the `kvb-skill-tune` skill's **Objective** section. Two figures
|
||||
with their residuals replace a hedging paragraph and save the operator a
|
||||
|
||||
@@ -0,0 +1,63 @@
|
||||
---
|
||||
name: kvb-check-gate
|
||||
description: >
|
||||
Prove 90/5: every `make test*` target green on the exact HEAD. Trigger
|
||||
before declaring 90/5 or labelling a PR, or on "gate status".
|
||||
---
|
||||
|
||||
The gate rule, stated here and nowhere else: nothing is declared 90/5 (fix
|
||||
confidence of at least 90 %, break risk under 5 %) before every `make test*`
|
||||
target of the project, each Makefile target named `test` or starting with
|
||||
`test-`, ran green on the exact head with a clean tracked tree. Until then
|
||||
the numbers are provisional and every target not green is a residual.
|
||||
|
||||
`${CLAUDE_SKILL_DIR}/scripts/gate.py` is the deterministic part. It reads the
|
||||
targets from the Makefile and every file it includes without running make,
|
||||
and keeps the results per commit in `kvb-gate.json` in the git common dir:
|
||||
linked worktrees share them, a new commit starts from zero.
|
||||
|
||||
## Steps
|
||||
|
||||
1. **List**: `gate.py list --repo <root>`. A warning about an unresolved
|
||||
include or target expression means the list may be short: read that line
|
||||
and name the gap. A project without any target keeps the gate open; ask
|
||||
the operator.
|
||||
2. **Status**: `gate.py status --repo <root>`. `gate: green` ends here.
|
||||
3. **Admit**: every gate target is heavy (`make test` and its siblings run
|
||||
whole suites, often containers). Ask the resource chair of the
|
||||
`kvb-org-organize` skill per target (what, where, how long, disk), wait
|
||||
for the grant, post the release to its news. Only a single test module
|
||||
self-admits (`${CLAUDE_SKILL_DIR}/../kvb-org-organize/scripts/watch.py
|
||||
admit --kind test-nostack`), and it never turns a gate target green. A
|
||||
hotfix takes the next free gate slot.
|
||||
4. **Run** the missing and red targets, tracked tree clean, in the
|
||||
background; give the operator `tail -f` on the log:
|
||||
|
||||
```bash
|
||||
gate.py run --repo <root> --target <t> [--target ...] --log-dir /tmp/<slug> [--fail-regex <re> ...]
|
||||
```
|
||||
|
||||
Pass one `--fail-regex` per failure line the project prints while exiting
|
||||
0; infinito-nexus needs `FAILED TARGETS: \S`. A target is red on a
|
||||
non-zero exit, a `make: ***` line, a matching regex, or tracked files it
|
||||
modified; the last one stops the run.
|
||||
5. **Judge** every red target by the line `status` prints and the log around
|
||||
it: caused by the change, already red on the default branch, or the
|
||||
environment. Fix the cause and run again; a red target never counts as
|
||||
green because it looks unrelated.
|
||||
6. **Status** once more; its last line is the verdict.
|
||||
|
||||
## Rules
|
||||
|
||||
- Only `gate.py run` on this head marks a target green. A peer's report, a
|
||||
CI run, an older commit or a run on a dirty tree never does.
|
||||
- 90/5 is allowed only on `gate: green` for the exact head: in the
|
||||
`kvb-check-confidence` block, the `90/5` label of the `kvb-git-pr-label`
|
||||
skill and the bar of the `kvb-org-split` skill.
|
||||
- Never edit `kvb-gate.json` by hand.
|
||||
|
||||
## Output
|
||||
|
||||
| Target | Status | Log | Line |
|
||||
|
||||
Then the `gate:` line verbatim.
|
||||
Executable
+345
@@ -0,0 +1,345 @@
|
||||
#!/usr/bin/env python3
|
||||
"""List, run and check a project's `make test*` targets against the current HEAD.
|
||||
|
||||
Args:
|
||||
list --repo <dir> [--json]: the Makefile targets named `test` or `test-*`,
|
||||
parsed statically from the Makefile make reads in <dir> and every file
|
||||
it includes; make is never run.
|
||||
status --repo <dir> [--json]: per listed target green, red or missing for
|
||||
HEAD, from `kvb-gate.json` in the git common dir.
|
||||
run --repo <dir> --target <t> [--target ...] --log-dir <dir>
|
||||
[--fail-regex <re> ...]: run each target as `make -C <dir> <t>` with
|
||||
stdout and stderr in <log-dir>/gate-<t>-<sha8>.log and record the
|
||||
result for HEAD. A target is red on a non-zero exit, a `make: ***`
|
||||
line, a matching --fail-regex, or tracked files it modified.
|
||||
|
||||
Output:
|
||||
list: one `<target>\t<file>:<line>` per target.
|
||||
status: one `<status>\t<target>\t<log>\t<finished>\t<reason>` per target,
|
||||
then `gate: green`, or `gate: open (...)` naming what is missing.
|
||||
run: one `<status>\t<target>\t<log>\t<reason>` per target.
|
||||
--json prints the same data as JSON.
|
||||
|
||||
Exit codes:
|
||||
list: 0 done, 2 usage error.
|
||||
status: 0 gate green, 1 gate open, 2 usage error.
|
||||
run: 0 every target green, 1 a target red, 2 usage error, dirty tracked
|
||||
files or HEAD moved during a run.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import fcntl
|
||||
import glob
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
from typing import NoReturn
|
||||
|
||||
MAKEFILES = ("GNUmakefile", "makefile", "Makefile")
|
||||
INCLUDE = re.compile(r"^\s*(?P<kind>-include|sinclude|include)\s+(?P<paths>.+)$")
|
||||
DEFINE = re.compile(r"^\s*(?:(?:export|override|private)\s+)*define\b")
|
||||
ENDEF = re.compile(r"^\s*endef\b")
|
||||
CONDITIONAL = re.compile(r"^\s*(?:ifeq|ifneq|ifdef|ifndef|else|endif)\b")
|
||||
ASSIGNMENT = re.compile(
|
||||
r"^\s*(?:(?:export|override|private|unexport)\s+)*[^\s:#=]+\s*(?::{1,3}=|[?+!]?=)"
|
||||
)
|
||||
COMMENT = re.compile(r"(?<!\\)#.*$")
|
||||
MAKE_ERROR = re.compile(r"\bmake(?:\[\d+\])?: \*\*\*")
|
||||
UNSAFE = re.compile(r"[^\w.-]")
|
||||
LINE_CHARS = 300
|
||||
STATE = "kvb-gate.json"
|
||||
|
||||
|
||||
def fail(message: str) -> NoReturn:
|
||||
print(f"gate: {message}", file=sys.stderr)
|
||||
sys.exit(2)
|
||||
|
||||
|
||||
def logical_lines(path: Path) -> list[tuple[int, str]]:
|
||||
lines: list[tuple[int, str]] = []
|
||||
start, buffer = 0, ""
|
||||
for number, raw in enumerate(path.read_text(encoding="utf-8", errors="replace").splitlines(), 1):
|
||||
if not buffer:
|
||||
start = number
|
||||
if (len(raw) - len(raw.rstrip("\\"))) % 2:
|
||||
buffer += raw[:-1] + " "
|
||||
continue
|
||||
lines.append((start, buffer + raw))
|
||||
buffer = ""
|
||||
if buffer:
|
||||
lines.append((start, buffer))
|
||||
return lines
|
||||
|
||||
|
||||
def rule_head(line: str) -> tuple[str, str] | None:
|
||||
depth = 0
|
||||
for index, char in enumerate(line):
|
||||
if char in "({":
|
||||
depth += 1
|
||||
elif char in ")}":
|
||||
depth = max(depth - 1, 0)
|
||||
elif depth == 0 and char == "=":
|
||||
return None
|
||||
elif depth == 0 and char == ":":
|
||||
rest = line[index:]
|
||||
if re.match(r":{1,3}=", rest):
|
||||
return None
|
||||
width = 2 if rest.startswith("::") else 1
|
||||
return line[:index].rstrip("&"), line[index + width:]
|
||||
return None
|
||||
|
||||
|
||||
def is_test(name: str) -> bool:
|
||||
return name == "test" or (name.startswith("test-") and "%" not in name)
|
||||
|
||||
|
||||
def parse(path: Path, repo: Path, found: dict, seen: set) -> None:
|
||||
resolved = path.resolve()
|
||||
if resolved in seen:
|
||||
return
|
||||
seen.add(resolved)
|
||||
shown = os.path.relpath(resolved, repo)
|
||||
define_depth = 0
|
||||
for number, line in logical_lines(path):
|
||||
if define_depth:
|
||||
if DEFINE.match(line):
|
||||
define_depth += 1
|
||||
elif ENDEF.match(line):
|
||||
define_depth -= 1
|
||||
continue
|
||||
if line.startswith("\t"):
|
||||
continue
|
||||
if DEFINE.match(line):
|
||||
define_depth = 1
|
||||
continue
|
||||
line = COMMENT.sub("", line)
|
||||
if not line.strip() or CONDITIONAL.match(line):
|
||||
continue
|
||||
include = INCLUDE.match(line)
|
||||
if include:
|
||||
for word in include.group("paths").split():
|
||||
if "$" in word:
|
||||
print(f"gate: {shown}:{number}: include {word} not resolved", file=sys.stderr)
|
||||
continue
|
||||
matches = sorted(glob.glob(str(repo / os.path.expanduser(word))))
|
||||
if not matches and include.group("kind") == "include":
|
||||
print(f"gate: {shown}:{number}: include {word} not found", file=sys.stderr)
|
||||
for match in matches:
|
||||
parse(Path(match), repo, found, seen)
|
||||
continue
|
||||
if ASSIGNMENT.match(line):
|
||||
continue
|
||||
head = rule_head(line)
|
||||
if head is None or ASSIGNMENT.match(head[1]):
|
||||
continue
|
||||
for name in head[0].split():
|
||||
if "$" in name:
|
||||
print(f"gate: {shown}:{number}: target {name} not resolved", file=sys.stderr)
|
||||
elif is_test(name) and name not in found:
|
||||
found[name] = f"{shown}:{number}"
|
||||
|
||||
|
||||
def targets(repo: Path) -> dict[str, str]:
|
||||
makefile = next((repo / n for n in MAKEFILES if (repo / n).is_file()), None)
|
||||
if makefile is None:
|
||||
fail(f"{repo} has no {', '.join(MAKEFILES)}")
|
||||
found: dict[str, str] = {}
|
||||
parse(makefile, repo, found, set())
|
||||
return dict(sorted(found.items()))
|
||||
|
||||
|
||||
def git(repo: Path, *args: str) -> str:
|
||||
result = subprocess.run(
|
||||
["git", *args], cwd=repo, capture_output=True, text=True, check=False
|
||||
)
|
||||
if result.returncode != 0:
|
||||
fail(f"git {' '.join(args)}: {result.stderr.strip()}")
|
||||
return result.stdout
|
||||
|
||||
|
||||
def head(repo: Path) -> str:
|
||||
return git(repo, "rev-parse", "--verify", "HEAD^{commit}").strip()
|
||||
|
||||
|
||||
def tree(repo: Path) -> tuple[list[str], list[str]]:
|
||||
dirty, untracked = [], []
|
||||
for line in git(repo, "status", "--porcelain=v1", "--untracked-files=all").splitlines():
|
||||
(untracked if line.startswith("??") else dirty).append(line[3:])
|
||||
return dirty, untracked
|
||||
|
||||
|
||||
def state_path(repo: Path) -> Path:
|
||||
return Path(git(repo, "rev-parse", "--path-format=absolute", "--git-common-dir").strip()) / STATE
|
||||
|
||||
|
||||
def load(path: Path) -> dict:
|
||||
if not path.exists():
|
||||
return {}
|
||||
try:
|
||||
data = json.loads(path.read_text(encoding="utf-8"))
|
||||
except ValueError:
|
||||
fail(f"{path} is not valid JSON; move it aside to start over")
|
||||
if not isinstance(data, dict):
|
||||
fail(f"{path} holds no object; move it aside to start over")
|
||||
return data
|
||||
|
||||
|
||||
def record(path: Path, sha: str, target: str, entry: dict) -> None:
|
||||
# Linked worktrees share the git common dir, so parallel runs write this file.
|
||||
with open(path.with_name(f"{STATE}.lock"), "w", encoding="utf-8") as lock:
|
||||
fcntl.flock(lock, fcntl.LOCK_EX)
|
||||
data = load(path)
|
||||
data.setdefault(sha, {})[target] = entry
|
||||
handle, temp = tempfile.mkstemp(dir=path.parent, prefix=f".{STATE}.")
|
||||
with os.fdopen(handle, "w", encoding="utf-8") as out:
|
||||
json.dump(data, out, indent=1, sort_keys=True)
|
||||
out.write("\n")
|
||||
os.replace(temp, path)
|
||||
|
||||
|
||||
def first_failure(log: Path, checks: list[re.Pattern]) -> str | None:
|
||||
with open(log, encoding="utf-8", errors="replace") as handle:
|
||||
for line in handle:
|
||||
if MAKE_ERROR.search(line) or any(check.search(line) for check in checks):
|
||||
return line.strip()[:LINE_CHARS]
|
||||
return None
|
||||
|
||||
|
||||
def repository(value: str) -> Path:
|
||||
repo = Path(value).resolve()
|
||||
if not repo.is_dir():
|
||||
fail(f"{value} is no directory")
|
||||
return repo
|
||||
|
||||
|
||||
def cmd_list(args: argparse.Namespace) -> int:
|
||||
found = targets(repository(args.repo))
|
||||
if args.json:
|
||||
print(json.dumps([{"target": t, "defined": d} for t, d in found.items()], indent=1))
|
||||
return 0
|
||||
for target, defined in found.items():
|
||||
print(f"{target}\t{defined}")
|
||||
return 0
|
||||
|
||||
|
||||
def cell(value: str | None) -> str:
|
||||
return "-" if value is None else value
|
||||
|
||||
|
||||
def cmd_status(args: argparse.Namespace) -> int:
|
||||
repo = repository(args.repo)
|
||||
found = targets(repo)
|
||||
sha = head(repo)
|
||||
state = load(state_path(repo))
|
||||
dirty, untracked = tree(repo)
|
||||
rows = []
|
||||
for target in found:
|
||||
entry = state[sha].get(target) if sha in state else None
|
||||
rows.append({"target": target, "status": "missing"} if entry is None else {"target": target, **entry})
|
||||
missing = [r["target"] for r in rows if r["status"] == "missing"]
|
||||
red = [r["target"] for r in rows if r["status"] == "red"]
|
||||
reasons = []
|
||||
if not found:
|
||||
reasons.append("no make test* target found")
|
||||
if missing:
|
||||
reasons.append(f"missing: {', '.join(missing)}")
|
||||
if red:
|
||||
reasons.append(f"red: {', '.join(red)}")
|
||||
if dirty:
|
||||
reasons.append(f"dirty: {len(dirty)} tracked file(s)")
|
||||
gate = "open" if reasons else "green"
|
||||
if args.json:
|
||||
print(json.dumps(
|
||||
{"sha": sha, "gate": gate, "open": reasons, "targets": rows,
|
||||
"dirty": dirty, "untracked": untracked},
|
||||
indent=1,
|
||||
))
|
||||
else:
|
||||
print(f"head\t{sha}")
|
||||
for r in rows:
|
||||
print("\t".join([r["status"], r["target"], *(cell(r.get(k)) for k in ("log", "finished", "reason"))]))
|
||||
if untracked:
|
||||
print(f"untracked\t{len(untracked)} file(s), not part of the gate")
|
||||
print("gate: green" if gate == "green" else f"gate: open ({'; '.join(reasons)})")
|
||||
return 0 if gate == "green" else 1
|
||||
|
||||
|
||||
def cmd_run(args: argparse.Namespace) -> int:
|
||||
repo = repository(args.repo)
|
||||
checks = []
|
||||
if args.fail_regex is not None:
|
||||
try:
|
||||
checks = [re.compile(pattern) for pattern in args.fail_regex]
|
||||
except re.error as error:
|
||||
fail(f"--fail-regex: {error}")
|
||||
found = targets(repo)
|
||||
unknown = [t for t in args.target if t not in found]
|
||||
if unknown:
|
||||
fail(f"not a make test* target of {repo}: {', '.join(unknown)}")
|
||||
sha = head(repo)
|
||||
dirty, untracked = tree(repo)
|
||||
if dirty:
|
||||
fail(f"tracked files are dirty, commit or drop them first: {', '.join(dirty)}")
|
||||
if untracked:
|
||||
print(f"untracked\t{len(untracked)} file(s), not part of the gate")
|
||||
log_dir = Path(args.log_dir).resolve()
|
||||
log_dir.mkdir(parents=True, exist_ok=True)
|
||||
state = state_path(repo)
|
||||
red = False
|
||||
for target in dict.fromkeys(args.target):
|
||||
log = log_dir / f"gate-{UNSAFE.sub('_', target)}-{sha[:8]}.log"
|
||||
with open(log, "w", encoding="utf-8") as out:
|
||||
code = subprocess.run(
|
||||
["make", "-C", str(repo), target],
|
||||
stdin=subprocess.DEVNULL, stdout=out, stderr=subprocess.STDOUT, check=False,
|
||||
).returncode
|
||||
if head(repo) != sha:
|
||||
fail(f"HEAD moved during {target}; nothing recorded for it")
|
||||
changed, _ = tree(repo)
|
||||
reasons = []
|
||||
if code:
|
||||
reasons.append(f"exit {code}")
|
||||
line = first_failure(log, checks)
|
||||
if line is not None:
|
||||
reasons.append(line)
|
||||
if changed:
|
||||
reasons.append(f"modified tracked files: {', '.join(changed)}")
|
||||
status = "red" if reasons else "green"
|
||||
red = red or status == "red"
|
||||
entry = {"status": status, "log": str(log),
|
||||
"finished": datetime.now(UTC).isoformat(timespec="seconds"),
|
||||
"fail_regex": [c.pattern for c in checks]}
|
||||
if reasons:
|
||||
entry["reason"] = "; ".join(reasons)
|
||||
record(state, sha, target, entry)
|
||||
print("\t".join([status, target, str(log), cell(entry.get("reason"))]))
|
||||
if changed:
|
||||
fail(f"{target} left tracked files dirty; remaining targets not run")
|
||||
return 1 if red else 0
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
|
||||
sub = parser.add_subparsers(dest="command", required=True)
|
||||
for name in ("list", "status"):
|
||||
command = sub.add_parser(name)
|
||||
command.add_argument("--repo", required=True)
|
||||
command.add_argument("--json", action="store_true")
|
||||
run = sub.add_parser("run")
|
||||
run.add_argument("--repo", required=True)
|
||||
run.add_argument("--target", action="append", required=True)
|
||||
run.add_argument("--log-dir", required=True)
|
||||
run.add_argument("--fail-regex", action="append")
|
||||
args = parser.parse_args()
|
||||
return {"list": cmd_list, "status": cmd_status, "run": cmd_run}[args.command](args)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -14,8 +14,8 @@ holds:
|
||||
- **`90/5`**: the pull request meets the bar of the `kvb-org-split` skill (fix
|
||||
confidence of at least 90 %, break risk under 5 %), from the
|
||||
`kvb-check-confidence` numbers in its body or freshly assessed, and only
|
||||
after the project's full quality gate passed on the PR's exact head; a
|
||||
hotfix takes the next free gate slot.
|
||||
when `gate.py status` of the `kvb-check-gate` skill prints `gate: green`
|
||||
with the PR's exact head checked out.
|
||||
|
||||
## Procedure
|
||||
|
||||
|
||||
@@ -0,0 +1,227 @@
|
||||
"""Exercise gate.py: static `make test*` listing, runs judged per target, and the per-HEAD gate."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from typing import ClassVar
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parents[1]
|
||||
SCRIPT = REPO_ROOT / "skills" / "kvb-check-gate" / "scripts" / "gate.py"
|
||||
GREEN = {
|
||||
"Makefile": (
|
||||
"include mk/*.mk\n"
|
||||
"-include absent.mk\n"
|
||||
"include $(EXTRA_MK)\n"
|
||||
".PHONY: test build\n"
|
||||
"VERSION := 1:2\n"
|
||||
"test: test-green\n"
|
||||
"\t@echo all green\n"
|
||||
"build:\n"
|
||||
"\t@echo build\n"
|
||||
"test-%:\n"
|
||||
"\t@echo pattern $@\n"
|
||||
"define BLOCK\n"
|
||||
"test-in-define:\n"
|
||||
"\t@echo never\n"
|
||||
"endef\n"
|
||||
),
|
||||
"mk/green.mk": (
|
||||
"test-green:\n"
|
||||
"\t@echo green\n"
|
||||
"test-green: FOO = bar\n"
|
||||
"test-only-var: FOO = bar\n"
|
||||
),
|
||||
"tracked.txt": "one\n",
|
||||
}
|
||||
RED = {
|
||||
"Makefile": (
|
||||
".PHONY: test test-red test-quiet test-swallow test-dirty\n"
|
||||
"test:\n"
|
||||
"\t@echo fine\n"
|
||||
"test-red:\n"
|
||||
"\t@echo boom\n"
|
||||
"\t@exit 1\n"
|
||||
"test-quiet:\n"
|
||||
'\t@echo "FAILED TARGETS: test-x"\n'
|
||||
"test-swallow:\n"
|
||||
"\t-@$(MAKE) --no-print-directory test-red\n"
|
||||
"test-dirty:\n"
|
||||
"\t@echo changed >> tracked.txt\n"
|
||||
),
|
||||
"tracked.txt": "one\n",
|
||||
}
|
||||
USAGE = {"Makefile": "test:\n\t@true\nbuild:\n\t@true\n"}
|
||||
FAILED = ("--fail-regex", r"FAILED TARGETS: \S")
|
||||
|
||||
|
||||
class GateCase(unittest.TestCase):
|
||||
files: ClassVar[dict[str, str]]
|
||||
|
||||
def setUp(self):
|
||||
tmp = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(tmp.cleanup)
|
||||
self.root = Path(tmp.name).resolve()
|
||||
self.repo = self.root / "repo"
|
||||
self.logs = self.root / "logs"
|
||||
self.repo.mkdir()
|
||||
self.env = {
|
||||
"PATH": "/usr/bin:/bin",
|
||||
"HOME": tmp.name,
|
||||
"GIT_CONFIG_NOSYSTEM": "1",
|
||||
"GIT_AUTHOR_NAME": "t",
|
||||
"GIT_AUTHOR_EMAIL": "t@example.org",
|
||||
"GIT_COMMITTER_NAME": "t",
|
||||
"GIT_COMMITTER_EMAIL": "t@example.org",
|
||||
}
|
||||
self._git("init", "-q", "-b", "main")
|
||||
for name, text in self.files.items():
|
||||
path = self.repo / name
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(text, encoding="utf-8")
|
||||
self._git("add", ".")
|
||||
self._git("commit", "-q", "-m", "init")
|
||||
|
||||
def _git(self, *args: str) -> str:
|
||||
return subprocess.run(
|
||||
["git", *args], cwd=self.repo, env=self.env, capture_output=True, text=True, check=True
|
||||
).stdout
|
||||
|
||||
def gate(self, *argv: str) -> subprocess.CompletedProcess:
|
||||
return subprocess.run(
|
||||
[sys.executable, str(SCRIPT), *argv], env=self.env, capture_output=True, text=True, check=False
|
||||
)
|
||||
|
||||
def run_targets(self, *targets: str, extra: tuple[str, ...]) -> subprocess.CompletedProcess:
|
||||
flags = [part for target in targets for part in ("--target", target)]
|
||||
return self.gate("run", "--repo", str(self.repo), *flags, "--log-dir", str(self.logs), *extra)
|
||||
|
||||
def status(self, repo: Path) -> tuple[int, dict]:
|
||||
result = self.gate("status", "--repo", str(repo), "--json")
|
||||
self.assertIn(result.returncode, (0, 1), result.stderr)
|
||||
return result.returncode, json.loads(result.stdout)
|
||||
|
||||
def last_status_line(self) -> str:
|
||||
return self.gate("status", "--repo", str(self.repo)).stdout.splitlines()[-1]
|
||||
|
||||
|
||||
class TestGreenPath(GateCase):
|
||||
files = GREEN
|
||||
|
||||
def test_list_reads_includes_and_skips_non_test_names(self):
|
||||
result = self.gate("list", "--repo", str(self.repo), "--json")
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertEqual(
|
||||
json.loads(result.stdout),
|
||||
[{"target": "test", "defined": "Makefile:6"}, {"target": "test-green", "defined": "mk/green.mk:1"}],
|
||||
)
|
||||
self.assertIn("include $(EXTRA_MK) not resolved", result.stderr)
|
||||
self.assertNotIn("absent.mk", result.stderr)
|
||||
|
||||
def test_gate_turns_green_and_a_new_commit_reopens_it(self):
|
||||
code, before = self.status(self.repo)
|
||||
self.assertEqual((code, before["gate"], before["open"]), (1, "open", ["missing: test, test-green"]))
|
||||
self.assertEqual(self.last_status_line(), "gate: open (missing: test, test-green)")
|
||||
|
||||
result = self.run_targets("test", "test-green", extra=())
|
||||
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
|
||||
log = self.logs / f"gate-test-{self._git('rev-parse', 'HEAD')[:8]}.log"
|
||||
self.assertEqual(result.stdout.splitlines()[0], f"green\ttest\t{log}\t-")
|
||||
self.assertIn("all green", log.read_text(encoding="utf-8"))
|
||||
|
||||
code, after = self.status(self.repo)
|
||||
self.assertEqual((code, after["gate"]), (0, "green"))
|
||||
self.assertEqual({r["target"]: r["status"] for r in after["targets"]}, {"test": "green", "test-green": "green"})
|
||||
self.assertEqual(self.last_status_line(), "gate: green")
|
||||
|
||||
self._git("commit", "-q", "--allow-empty", "-m", "next")
|
||||
code, reopened = self.status(self.repo)
|
||||
self.assertEqual((code, reopened["open"]), (1, ["missing: test, test-green"]))
|
||||
|
||||
def test_results_are_shared_by_linked_worktrees(self):
|
||||
self.assertEqual(self.run_targets("test", "test-green", extra=()).returncode, 0)
|
||||
worktree = self.root / "wt"
|
||||
self._git("worktree", "add", "-q", "--detach", str(worktree))
|
||||
self.assertEqual(self.status(worktree)[0], 0)
|
||||
|
||||
def test_dirty_tracked_file_blocks_run_untracked_is_reported(self):
|
||||
(self.repo / "scratch.txt").write_text("x\n", encoding="utf-8")
|
||||
result = self.run_targets("test-green", extra=())
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertIn("untracked\t1 file(s), not part of the gate", result.stdout)
|
||||
|
||||
(self.repo / "tracked.txt").write_text("two\n", encoding="utf-8")
|
||||
result = self.run_targets("test", extra=())
|
||||
self.assertEqual(result.returncode, 2)
|
||||
self.assertIn("tracked files are dirty", result.stderr)
|
||||
self.assertIn("tracked.txt", result.stderr)
|
||||
code, status = self.status(self.repo)
|
||||
self.assertEqual((code, status["open"]), (1, ["missing: test", "dirty: 1 tracked file(s)"]))
|
||||
|
||||
|
||||
class TestRedPath(GateCase):
|
||||
files = RED
|
||||
|
||||
def results(self) -> dict[str, dict]:
|
||||
return {r["target"]: r for r in self.status(self.repo)[1]["targets"]}
|
||||
|
||||
def test_exit_code_fail_regex_and_swallowed_make_error_are_red(self):
|
||||
result = self.run_targets("test", "test-red", "test-quiet", "test-swallow", extra=FAILED)
|
||||
self.assertEqual(result.returncode, 1, result.stderr)
|
||||
rows = self.results()
|
||||
self.assertEqual(rows["test"]["status"], "green")
|
||||
self.assertEqual(rows["test-red"]["status"], "red")
|
||||
self.assertRegex(rows["test-red"]["reason"], r"^exit 2; make: \*\*\* \[Makefile:\d+: test-red\] Error 1$")
|
||||
self.assertEqual(rows["test-quiet"]["status"], "red")
|
||||
self.assertEqual(rows["test-quiet"]["reason"], "FAILED TARGETS: test-x")
|
||||
self.assertEqual(rows["test-quiet"]["fail_regex"], [FAILED[1]])
|
||||
self.assertEqual(rows["test-swallow"]["status"], "red")
|
||||
self.assertRegex(rows["test-swallow"]["reason"], r"^make\[1\]: \*\*\* ")
|
||||
self.assertEqual(rows["test-dirty"]["status"], "missing")
|
||||
self.assertEqual(
|
||||
self.last_status_line(), "gate: open (missing: test-dirty; red: test-quiet, test-red, test-swallow)"
|
||||
)
|
||||
|
||||
def test_a_fail_pattern_without_its_regex_stays_green(self):
|
||||
self.assertEqual(self.run_targets("test-quiet", extra=()).returncode, 0)
|
||||
self.assertEqual(self.results()["test-quiet"]["status"], "green")
|
||||
|
||||
def test_a_target_that_modifies_tracked_files_is_red_and_stops_the_run(self):
|
||||
result = self.run_targets("test-dirty", "test", extra=())
|
||||
self.assertEqual(result.returncode, 2)
|
||||
self.assertIn("test-dirty left tracked files dirty", result.stderr)
|
||||
rows = self.results()
|
||||
self.assertEqual(rows["test-dirty"]["reason"], "modified tracked files: tracked.txt")
|
||||
self.assertEqual(rows["test"]["status"], "missing")
|
||||
|
||||
|
||||
class TestUsage(GateCase):
|
||||
files = USAGE
|
||||
|
||||
def test_usage_errors_exit_2(self):
|
||||
bare = self.root / "bare"
|
||||
bare.mkdir()
|
||||
cases = {
|
||||
"no command": self.gate(),
|
||||
"run without log dir": self.gate("run", "--repo", str(self.repo), "--target", "test"),
|
||||
"run without target": self.gate("run", "--repo", str(self.repo), "--log-dir", str(self.logs)),
|
||||
"non-test target": self.run_targets("build", extra=()),
|
||||
"unknown target": self.run_targets("test-nope", extra=()),
|
||||
"bad regex": self.run_targets("test", extra=("--fail-regex", "(")),
|
||||
"no Makefile": self.gate("list", "--repo", str(bare)),
|
||||
"missing repo": self.gate("status", "--repo", str(self.root / "absent")),
|
||||
}
|
||||
(bare / "Makefile").write_text("test:\n\t@true\n", encoding="utf-8")
|
||||
cases["not a git repository"] = self.gate("status", "--repo", str(bare))
|
||||
for name, result in cases.items():
|
||||
with self.subTest(case=name):
|
||||
self.assertEqual(result.returncode, 2, result.stdout + result.stderr)
|
||||
self.assertFalse(self.logs.exists())
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
Reference in new issue
Block a user