feat(check): security value, commit tables, MEDUSA install and kvb-think-choice

kvb-check-security gains security.py: it scans the changed files with MEDUSA, weights findings (critical -50, high -25, medium -10, low -2) and prints locations and counts only; without MEDUSA it asks for a review by hand passed as '~<n>'. kvb-check-score takes --security and --critical: the score multiplies by security/100, a critical finding blocks the commit (exit 4), security below 100 asks the operator like efficiency or speed over 100, and a 90/5 failure now lists every cause. --table and --commit-table print the values as markdown tables, shown before and after every commit.

make install installs MEDUSA with pipx unless it is on the PATH. The new kvb-think-choice skill offers open decisions as multiple choice and checkboxes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Kevin Veen-BirkenbachandClaude Opus 5.5 committed 2026-11-04 17:48:42 +01:00
1 parent 0343a2f3ce
commit 62a0bcf7e9
12 files changed
+786 -123

No files matched your search

+3 -1
View File
@@ -81,6 +81,7 @@ flowchart LR
org --> org_evening[kvb-org-evening]
org --> org_sunset[kvb-org-sunset]
org --> org_sunrise[kvb-org-sunrise]
think --> think_choice[kvb-think-choice]
org --> org_radio[kvb-org-radio]
org --> org_queue[kvb-org-queue]
agent --> agent_optimize[kvb-agent-optimize]
@@ -125,6 +126,7 @@ flowchart LR
criticize[kvb-think-criticize]
brainstorm[kvb-think-brainstorm]
gaps[kvb-think-gaps]
choice[kvb-think-choice]
end
subgraph cat_create [create]
plan[kvb-create-plan]
@@ -380,7 +382,7 @@ Copy the skills into a specific project (`<repo>/.agents/skills` and `<repo>/.cl
make project TARGET=/path/to/repo
```
Both commands also patch the target's `.claude/settings.local.json` — the machine-local, untracked file, so a project's reviewed `.claude/settings.json` is never rewritten by an install: the caveman and ponytail plugins are enabled so their modes auto-activate on session start, and two reminder hooks are registered — `kvb-dream` points each agent at `/kvb-dream` at most once every twelve hours (stamped per session in `~/.claude/.dream-stamps/`), and `kvb-coach` runs its short form at the first prompt of every full hour, once across all agents (stamped in `~/.claude/.coach-hours/`); the global install also enables the `kvb-dream-notify.timer` systemd user timer, which recommends `/kvb-dream` in a GNOME notification daily at 17:00 local time and catches up once after a missed run. A PreToolUse gate from the `kvb-org-organize` skill blocks edits to files in a git worktree until the session holds a claim (`skills/kvb-org-organize/scripts/council.py`), and its resource watch (`skills/kvb-org-organize/scripts/watch.py`) measures disk, Docker and caches, cleans up low-risk data and blocks new heavy jobs before the disk runs out; it stays inactive until `~/.config/agents-council/config.json` exists. The `kvb-org-radio` skill registers three hooks: its prompt hook delivers `broadcast` and `direct` items on the next prompt, a PostToolBatch hook delivers `security`, `pan-pan` and `mayday` items mid-turn, and an `asyncRewake` Stop hook wakes an idle session for a `mayday` or a `direct` item sent with `--wake`. The PreToolUse hook of the `kvb-org-queue` skill sends heavy commands through `queue.py hold` once the config sets `queue_enforce`. Two hooks of the `kvb-org-perform` skill (`skills/kvb-org-perform/scripts/log.py`) log every Agent spawn with its task class, model and effort, and store the quality feedback a sandboxed session prints, in `~/.local/state/agents-council/subagent-model.jsonl`. Existing settings are preserved.
Both commands also patch the target's `.claude/settings.local.json` — the machine-local, untracked file, so a project's reviewed `.claude/settings.json` is never rewritten by an install: the caveman and ponytail plugins are enabled so their modes auto-activate on session start, and two reminder hooks are registered — `kvb-dream` points each agent at `/kvb-dream` at most once every twelve hours (stamped per session in `~/.claude/.dream-stamps/`), and `kvb-coach` runs its short form at the first prompt of every full hour, once across all agents (stamped in `~/.claude/.coach-hours/`); the global install also enables the `kvb-dream-notify.timer` systemd user timer, which recommends `/kvb-dream` in a GNOME notification daily at 17:00 local time and catches up once after a missed run. A PreToolUse gate from the `kvb-org-organize` skill blocks edits to files in a git worktree until the session holds a claim (`skills/kvb-org-organize/scripts/council.py`), and its resource watch (`skills/kvb-org-organize/scripts/watch.py`) measures disk, Docker and caches, cleans up low-risk data and blocks new heavy jobs before the disk runs out; it stays inactive until `~/.config/agents-council/config.json` exists. The `kvb-org-radio` skill registers three hooks: its prompt hook delivers `broadcast` and `direct` items on the next prompt, a PostToolBatch hook delivers `security`, `pan-pan` and `mayday` items mid-turn, and an `asyncRewake` Stop hook wakes an idle session for a `mayday` or a `direct` item sent with `--wake`. The PreToolUse hook of the `kvb-org-queue` skill sends heavy commands through `queue.py hold` once the config sets `queue_enforce`. The global install also installs the MEDUSA security scanner with pipx for `kvb-check-security`, unless `medusa` is already on the PATH. Two hooks of the `kvb-org-perform` skill (`skills/kvb-org-perform/scripts/log.py`) log every Agent spawn with its task class, model and effort, and store the quality feedback a sandboxed session prints, in `~/.local/state/agents-council/subagent-model.jsonl`. Existing settings are preserved.
Restart your agent afterwards so it loads the new skills.
+9
View File
@@ -73,6 +73,15 @@ if [[ "${TARGET}" == "${HOME}" ]]; then
else
warn "skills: codebase-memory-mcp is not runnable from PATH; the code graph stays off (see kvb-agent-optimize)."
fi
if command -v medusa &>/dev/null; then
log "skills: the MEDUSA security scanner is installed"
elif ! command -v pipx &>/dev/null; then
warn "skills: pipx not found; install the MEDUSA scanner with 'pip install --user medusa-security'. Until then kvb-check-security reviews by hand."
elif pipx install medusa-security; then
log "skills: installed the MEDUSA security scanner for kvb-check-security"
else
warn "skills: the MEDUSA install failed; kvb-check-security reviews by hand."
fi
fi
log "skills: done. Restart your agent to load the skills."
+6 -4
View File
@@ -15,11 +15,13 @@ and never estimates one itself.
**Break risk** (percent): probability it breaks anything else, from the
`kvb-check-break-risk` skill. Two numbers, never one blended figure.
2. **Score**: pass both, with efficiency from the `kvb-check-efficiency`
skill and speed from the `kvb-check-speed` skill, to
skill, speed from the `kvb-check-speed` skill and security and the
critical count from `security.py` of the `kvb-check-security` skill, to
`${CLAUDE_SKILL_DIR}/../kvb-check-score/scripts/score.py --fix <n> --break
<n> --efficiency <n|-> --speed <n|->` and show its score line and gate line
verbatim. Exit 1 fails the 90/5 rule; exit 3 means ask the operator again
before the commit, naming the cause the gate line prints.
<n> --efficiency <n|-> --speed <n|-> --security <n|~n> --critical <count>`
and show its score line and gate line verbatim. Exit 4 forbids the commit
(critical finding); exit 1 fails the 90/5 rule; exit 3 means ask the
operator again before the commit, naming the causes the gate line prints.
3. **Evidence**: the strongest proof lines only: what was executed,
reproduced, enumerated, or read from an authoritative source. Mark proven
(ran it, saw it) apart from derived (reasoned statically).
+27 -15
View File
@@ -1,7 +1,7 @@
---
name: kvb-check-score
description: >
Score fix, break, efficiency and speed; run wherever confidence is reported.
Score fix, break, efficiency, speed and security; run wherever confidence is reported.
---
Run it wherever fix confidence and break risk are reported: every
@@ -10,7 +10,7 @@ One script call prints two lines; paste them verbatim, they replace prose
numbers.
```bash
python3 ${CLAUDE_SKILL_DIR}/scripts/score.py --fix <0-100> --break <0-100> --efficiency <n|-|new> --speed <n|->
python3 ${CLAUDE_SKILL_DIR}/scripts/score.py --fix <0-100> --break <0-100> --efficiency <n|-|new> --speed <n|-> --security <n|~n> --critical <count>
```
| Emoji | Value | Source |
@@ -19,37 +19,49 @@ python3 ${CLAUDE_SKILL_DIR}/scripts/score.py --fix <0-100> --break <0-100> --eff
| 💥 | break risk | `kvb-check-break-risk` |
| 🌱 | efficiency | `tokens.py` (skills) or `measure.py` (code) of `kvb-check-efficiency` |
| ⚡ | speed | `measure.py`, see `kvb-check-speed` |
| 🛡️ | security, critical count | `security.py` of `kvb-check-security`; `~<n>` from a review by hand |
| 🏆 | score | this script |
`-` means not measured or not relevant (skills have no speed) and counts as
100; it shows as `–`. `new` (changed skill text without a base) counts as 100
in the score, yet still asks. Never invent a value: measure it or pass `-`.
Score = `fix * (100 - break) / 100 * 100 / efficiency * 100 / speed`,
Score = `fix * (100 - break) / 100 * 100 / efficiency * 100 / speed * security / 100`,
capped to 0..100 and rounded.
## Gate
The second line is the verdict:
The second line is the verdict, first match wins:
- `✅ gate: 90/5, efficiency and speed <= 100` (exit 0): commit as usual.
90/5 still needs `gate: green` of `kvb-check-gate`.
- `⛔ gate: critical security finding: do not commit` (exit 4): the commit
call is not sent, whatever else holds and whatever the operator approved
before. Fix the finding, rescan, score again.
- `⛔ gate: fix <f> % < 90` or `⛔ gate: break <b> % > 5` (exit 1): the 90/5
rule fails, which bars PR labels and the split bar, not a commit: show it
in the block and commit only on the operator's word. Close the gap as the
last section of `kvb-check-break-risk` says.
- `⚠️ gate: efficiency <e> % > 100: ask the operator before the commit`, the
same for speed (exit 3): the change costs more than its base. Before the
commit, ask the operator with both lines and what the extra cost buys;
commit only on a yes, which never carries over to another commit.
- `⚠️ gate: <causes>: ask the operator before the commit` (exit 3), naming
each of `efficiency <e> % > 100`, `speed <s> % > 100` and
`security <n> % < 100`: the change costs more than its base or carries
findings. Before the commit, ask the operator with the block and what the
cost buys or why each finding is acceptable; commit only on a yes, which
never carries over to another commit.
- `✅ gate: 90/5, efficiency and speed <= 100, security 100 %` (exit 0):
commit as usual. 90/5 still needs `gate: green` of `kvb-check-gate`.
- Exit 2: usage error.
## Commit blocks
## Tables
A commit block puts one line above the two:
`--table` prints the same values as one markdown table instead of the two
lines:
```text
📁 <worktree> · 🌿 <branch> · 📊 <git diff --cached --shortstat>
🎯 fix 95 % · 💥 break 2 % · 🌱 efficiency 97 % · ⚡ speed – · 🏆 score 96
✅ gate: 90/5, efficiency and speed <= 100
| 🎯 Fix | 💥 Break | 🌱 Efficiency | ⚡ Speed | 🛡️ Security | 🏆 Score | Gate |
```
`--commit-table <repo>` leads that table with `📁 Worktree | 🌿 Branch |
📊 Diff stat`, read from git: the top level, the current branch (or
`detached at <sha>`) and the staged `git diff --cached --shortstat` as
`<n> files +<added>/-<removed>`. Every commit block is this table. After the
commit, add `--commit <sha>`: the diff stat of that commit and a `🔖 Commit`
column.
+126 -24
View File
@@ -1,5 +1,5 @@
#!/usr/bin/env python3
"""Fold fix confidence, break risk, efficiency and speed into one score line and one gate line.
"""Fold fix confidence, break risk, efficiency, speed and security into one score and one gate verdict.
Args:
--fix: fix confidence in percent, 0 to 100.
@@ -7,29 +7,48 @@ Args:
--efficiency: efficiency in percent above 0; `-` when not measured or not
relevant, `new` when the change has no base. Both count as 100 in the score.
--speed: speed in percent above 0, `-` or `new` as for --efficiency.
--security: security in percent, 0 to 100, from security.py of kvb-check-security;
`~<n>` for a value from a review by hand.
--critical: count of critical security findings.
--table: print one markdown table instead of the two lines.
--commit-table <repo>: the same table, led by the worktree, branch and staged
diff stat of that repository.
--commit <sha>: with --commit-table, the diff stat of that commit instead of
the staged one, plus a commit column.
Output:
Line 1: `🎯 fix <f> % · 💥 break <b> % · 🌱 efficiency <e> % · ⚡ speed <s> % · 🏆 score <n>`,
`–` in place of `<x> %` for `-`, `new` for `new`. Score is
fix * (100 - break) / 100 * 100 / efficiency * 100 / speed, capped to 0..100, rounded half up.
Line 1: `🎯 fix <f> % · 💥 break <b> % · 🌱 efficiency <e> % · ⚡ speed <s> % ·
🛡️ security <n> % · 🏆 score <n>`, `–` in place of `<x> %` for `-`, `new` for `new`.
Score is fix * (100 - break) / 100 * 100 / efficiency * 100 / speed * security / 100,
capped to 0..100, rounded half up.
Line 2: the gate verdict, `✅ gate: ...`, `⛔ gate: ...` or `⚠️ gate: ...`.
Exit codes:
0 gate passed, 1 fix under 90 or break over 5, 2 usage error,
3 efficiency or speed over 100 or new: ask the operator before the commit.
3 efficiency or speed over 100 or new, or security under 100: ask the operator
before the commit, 4 a critical security finding: do not commit.
"""
from __future__ import annotations
import argparse
import math
import re
import subprocess
import sys
FIX_MIN = 90
BREAK_MAX = 5
COST_MAX = 100
SAFE = 100
DASH = "-"
NEW = "new"
ESTIMATE = "~"
SCORE_HEADER = ("🎯 Fix", "💥 Break", "🌱 Efficiency", "⚡ Speed", "🛡️ Security", "🏆 Score", "Gate")
COMMIT_HEADER = ("📁 Worktree", "🌿 Branch", "📊 Diff stat")
SHORTSTAT = re.compile(
r"(\d+) files? changed(?:, (\d+) insertions?\(\+\))?(?:, (\d+) deletions?\(-\))?"
)
def percent(text: str) -> float:
@@ -54,6 +73,17 @@ def cost(text: str) -> float | str:
return value
def security(text: str) -> tuple[float, str]:
mark = ESTIMATE if text.startswith(ESTIMATE) else ""
return percent(text.removeprefix(ESTIMATE)), mark
def count(text: str) -> int:
if not text.isdigit():
raise argparse.ArgumentTypeError(f"{text!r} is not a count")
return int(text)
def number(value: float) -> str:
return f"{value:g}"
@@ -70,8 +100,11 @@ def factor(value: float | str) -> float:
return 100 / value if isinstance(value, float) else 1.0
def score(fix: float, risk: float, efficiency: float | str, speed: float | str) -> int:
raw = fix * (100 - risk) / 100 * factor(efficiency) * factor(speed)
def score(args: argparse.Namespace) -> int:
raw = (
args.fix * (100 - args.risk) / 100 * factor(args.efficiency) * factor(args.speed)
* args.security[0] / 100
)
return math.floor(min(max(raw, 0.0), 100.0) + 0.5)
@@ -83,18 +116,67 @@ def over(name: str, value: float | str) -> str | None:
return None
def gate(fix: float, risk: float, efficiency: float | str, speed: float | str) -> tuple[str, int]:
def gate(args: argparse.Namespace) -> tuple[str, str, int]:
if args.critical > 0:
return "⛔", "critical security finding: do not commit", 4
failed = []
if fix < FIX_MIN:
failed.append(f"fix {number(fix)} % < {FIX_MIN}")
if risk > BREAK_MAX:
failed.append(f"break {number(risk)} % > {BREAK_MAX}")
if args.fix < FIX_MIN:
failed.append(f"fix {number(args.fix)} % < {FIX_MIN}")
if args.risk > BREAK_MAX:
failed.append(f"break {number(args.risk)} % > {BREAK_MAX}")
value, mark = args.security
unsafe = f"security {mark}{number(value)} % < {SAFE}" if value < SAFE else None
asks = [text for text in (over("efficiency", args.efficiency), over("speed", args.speed), unsafe) if text]
ask = f"{', '.join(asks)}: ask the operator before the commit" if asks else ""
if failed:
return f"⛔ gate: {', '.join(failed)}", 1
costly = [text for text in (over("efficiency", efficiency), over("speed", speed)) if text]
if costly:
return f"⚠️ gate: {', '.join(costly)}: ask the operator before the commit", 3
return f"✅ gate: {FIX_MIN}/{BREAK_MAX}, efficiency and speed <= {COST_MAX}", 0
return "⛔", "; ".join(part for part in (", ".join(failed), ask) if part), 1
if asks:
return "⚠️", ask, 3
return "✅", f"{FIX_MIN}/{BREAK_MAX}, efficiency and speed <= {COST_MAX}, security {SAFE} %", 0
def git(repo: str, *args: str) -> str:
try:
result = subprocess.run(["git", *args], cwd=repo, capture_output=True, text=True, check=False)
except OSError as error:
print(f"score: git {args[0]} in {repo}: {error}", file=sys.stderr)
sys.exit(2)
if result.returncode != 0:
print(f"score: git {args[0]} in {repo}: {result.stderr.strip()}", file=sys.stderr)
sys.exit(2)
return result.stdout.strip()
def diffstat(shortstat: str) -> str:
match = SHORTSTAT.search(shortstat)
files, added, removed = (int(g or 0) for g in match.groups()) if match else (0, 0, 0)
return f"{files} file{'' if files == 1 else 's'} +{added}/-{removed}"
def commit_cells(repo: str, commit: str | None) -> tuple[list[str], list[str]]:
branch = git(repo, "branch", "--show-current") or f"detached at {git(repo, 'rev-parse', '--short', 'HEAD')}"
if commit is None:
stat = git(repo, "diff", "--cached", "--shortstat")
return list(COMMIT_HEADER), [git(repo, "rev-parse", "--show-toplevel"), branch, diffstat(stat)]
stat = git(repo, "show", "--shortstat", "--format=", commit)
return (
[*COMMIT_HEADER, "🔖 Commit"],
[git(repo, "rev-parse", "--show-toplevel"), branch, diffstat(stat), git(repo, "rev-parse", "--short", commit)],
)
def cell(text: str) -> str:
return text.replace("|", "\\|")
def table(header: list[str], row: list[str]) -> str:
return "\n".join(
(
"| " + " | ".join(header) + " |",
"|" + " --- |" * len(header),
"| " + " | ".join(cell(value) for value in row) + " |",
)
)
def main() -> int:
@@ -103,14 +185,34 @@ def main() -> int:
parser.add_argument("--break", dest="risk", type=percent, required=True)
parser.add_argument("--efficiency", type=cost, required=True)
parser.add_argument("--speed", type=cost, required=True)
parser.add_argument("--security", type=security, required=True)
parser.add_argument("--critical", type=count, required=True)
layout = parser.add_mutually_exclusive_group()
layout.add_argument("--table", action="store_true")
layout.add_argument("--commit-table", metavar="REPO")
parser.add_argument("--commit", metavar="SHA")
args = parser.parse_args()
total = score(args.fix, args.risk, args.efficiency, args.speed)
print(
f"🎯 fix {number(args.fix)} % · 💥 break {number(args.risk)} % · "
f"🌱 efficiency {shown(args.efficiency)} · ⚡ speed {shown(args.speed)} · 🏆 score {total}"
)
verdict, code = gate(args.fix, args.risk, args.efficiency, args.speed)
print(verdict)
if args.commit and not args.commit_table:
parser.error("--commit needs --commit-table")
total = score(args)
emoji, verdict, code = gate(args)
value, mark = args.security
values = [
f"{number(args.fix)} %", f"{number(args.risk)} %", shown(args.efficiency), shown(args.speed),
f"{mark}{number(value)} %", str(total), f"{emoji} {verdict}",
]
if args.commit_table:
header, cells = commit_cells(args.commit_table, args.commit)
print(table([*header, *SCORE_HEADER], [*cells, *values]))
elif args.table:
print(table(list(SCORE_HEADER), values))
else:
print(
f"🎯 fix {values[0]} · 💥 break {values[1]} · 🌱 efficiency {values[2]} · "
f"⚡ speed {values[3]} · 🛡️ security {values[4]} · 🏆 score {total}"
)
print(f"{emoji} gate: {verdict}")
return code
+26 -1
View File
@@ -11,7 +11,7 @@ MEDUSA is a defensive AI-first security scanner (40,000+ patterns: CVEs, prompt
## Setup
Check availability with `medusa --version`.
If missing, install it with `pipx install medusa-security` (fallback: `pip install --user medusa-security`).
The skills repository's `make install` installs it with pipx; if it is still missing, `pipx install medusa-security` (fallback: `pip install --user medusa-security`) is the operator's step.
## Commands
@@ -22,6 +22,31 @@ If missing, install it with `pipx install medusa-security` (fallback: `pip insta
| Find leaked API keys in AI chat + shell history | `medusa secrets scan` |
| Scan one exported file | `medusa secrets scan --path <file>` |
## Security value of a change
The 🛡️ value of the `kvb-check-score` block, 100 % meaning safe:
```bash
python3 ${CLAUDE_SKILL_DIR}/scripts/security.py --repo <root> --base <ref>
```
It copies the files changed since `<ref>`, untracked ones included and
symlinks left out, to a temporary tree, scans it once with
`medusa scan <tree> --format json -o <dir> --no-cache` and prints one line:
`🛡️ security <n> % (medusa: <k> findings in <m> files: <severity> <path>:<line>, ...)`.
`<n>` is 100 minus 50 per critical, 25 per high, 10 per medium and 2 per low
finding, at least 0. It shows locations and counts only, never a matched
value. Pass `<n>` as `--security` and the critical count as `--critical`
(`--json` holds it) to `score.py`.
- Exit 0: 100 %. Exit 3: below 100 without a critical finding; the score
gate asks the operator. Exit 1: a critical finding; the score gate forbids
the commit.
- Exit 4: medusa is missing or its report unreadable. Review trust
boundaries, secrets and injection in the diff by hand, then pass
`--security '~<n>'` (quoted: zsh expands a bare `~100` as a user's home) and the critical findings you found as `--critical`.
- Exit 2: usage error.
## Rules
- Run scans read-only; never pass `--reveal` and never run `medusa secrets purge` unless the user explicitly asks for it.
+186
View File
@@ -0,0 +1,186 @@
#!/usr/bin/env python3
"""Scan the files a change touches with MEDUSA and fold the findings into one security percentage.
Args:
--repo: git repository whose working tree holds the change.
--base: git ref the change starts from.
--json: print the figures as JSON instead of the line.
Scan:
The regular files `git diff <ref>` names plus untracked ones, symlinks left
out, copied to a temporary tree and scanned once with
`medusa scan <tree> --format json -o <dir> --no-cache`, read from
`<dir>/medusa-scan-*.json`: a `findings` list whose entries carry
`severity` (critical, high, medium, low or info), `file` and `line`.
Output:
`🛡️ security <n> % (medusa: <k> findings in <m> files: <severity> <path>:<line>, ...)`,
m the files scanned, the first 5 locations by severity and `+<rest> more`.
n = 100 - 50 * critical - 25 * high - 10 * medium - 2 * low, at least 0. Matched
values are never printed. --json adds the counts per severity and every location.
Exit codes:
0 security 100, 1 a critical finding, 2 usage error, 3 under 100 without a
critical finding, 4 medusa missing or its report unreadable: review the diff by hand.
"""
from __future__ import annotations
import argparse
import json
import shutil
import subprocess
import sys
import tempfile
from pathlib import Path
WEIGHTS = {"critical": 50, "high": 25, "medium": 10, "low": 2, "info": 0}
SAFE = 100
SHOWN = 5
HAND = (
"review trust boundaries, secrets and injection in the diff by hand, "
"then pass --security '~<n>' to kvb-check-score"
)
class Unreadable(Exception):
pass
def fail(message: str) -> None:
print(f"security: {message}", file=sys.stderr)
sys.exit(2)
def git(repo: Path, *args: str) -> str:
result = subprocess.run(["git", *args], cwd=repo, capture_output=True, check=False)
if result.returncode != 0:
fail(result.stderr.decode(errors="replace").strip() or f"git {args[0]} failed in {repo}")
return result.stdout.decode(errors="replace")
def changed(root: Path, base: str) -> list[str]:
# Trip-wire: with rename detection a moved file lists only its new path.
listed = git(root, "diff", "--no-renames", "-z", "--name-only", base, "--")
listed += git(root, "ls-files", "-z", "--others", "--exclude-standard")
return sorted({p for p in listed.split("\0") if p})
def copy(root: Path, paths: list[str], tree: Path) -> int:
copied = 0
for name in paths:
source = root / name
# Trip-wire: a symlink's target may lie outside the repository; even a copied link lets the scan read it.
if source.is_symlink() or not source.is_file():
continue
target = tree / name
target.parent.mkdir(parents=True, exist_ok=True)
shutil.copy2(source, target)
copied += 1
return copied
def located(name: str, tree: Path) -> str:
path = Path(name)
if not path.is_absolute():
path = tree / path if (tree / path).exists() else tree.parent / path
try:
return path.resolve().relative_to(tree.resolve()).as_posix()
except ValueError:
return name
def findings(reports: Path, tree: Path) -> list[dict]:
files = sorted(reports.glob("medusa-scan-*.json")) if reports.is_dir() else []
if not files:
raise Unreadable(f"no medusa-scan-*.json in {reports}")
try:
report = json.loads(files[-1].read_text(encoding="utf-8"))
except (OSError, ValueError) as error:
raise Unreadable(f"{files[-1].name}: {error}") from None
entries = report.get("findings") if isinstance(report, dict) else None
if not isinstance(entries, list):
raise Unreadable(f"{files[-1].name} holds no findings list")
found = []
for entry in entries:
severity = str(entry.get("severity", "")).lower() if isinstance(entry, dict) else ""
if severity not in WEIGHTS:
raise Unreadable(f"{files[-1].name}: a finding without a known severity")
line = entry.get("line")
found.append({
"severity": severity,
"path": located(str(entry.get("file") or "?"), tree),
"line": line if isinstance(line, int) else None,
})
rank = list(WEIGHTS)
return sorted(found, key=lambda f: (rank.index(f["severity"]), f["path"], f["line"] or 0))
def scan(medusa: str, root: Path, base: str) -> tuple[int, list[dict]]:
with tempfile.TemporaryDirectory(prefix="kvb-security-") as scratch:
work = Path(scratch)
tree, reports = work / "tree", work / "reports"
tree.mkdir()
scanned = copy(root, changed(root, base), tree)
if not scanned:
return 0, []
result = subprocess.run(
[medusa, "scan", str(tree), "--format", "json", "-o", str(reports), "--no-cache"],
cwd=work, stdin=subprocess.DEVNULL, capture_output=True, check=False,
)
try:
return scanned, findings(reports, tree)
except Unreadable as error:
raise Unreadable(f"{error} (medusa exit {result.returncode})") from None
def plural(n: int, noun: str) -> str:
return f"{n} {noun}{'' if n == 1 else 's'}"
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
parser.add_argument("--repo", required=True)
parser.add_argument("--base", required=True)
parser.add_argument("--json", action="store_true")
args = parser.parse_args()
repo = Path(args.repo)
if not repo.is_dir():
fail(f"{repo} is not a directory")
root = Path(git(repo, "rev-parse", "--show-toplevel").strip())
if subprocess.run(
["git", "rev-parse", "--verify", "--quiet", f"{args.base}^{{commit}}"], cwd=root, capture_output=True, check=False
).returncode != 0:
fail(f"unknown ref {args.base!r}")
medusa = shutil.which("medusa")
problem = None if medusa else "medusa missing"
if medusa:
try:
scanned, found = scan(medusa, root, args.base)
except Unreadable as error:
problem = f"medusa report unreadable: {error}"
if problem:
if args.json:
print(json.dumps({"security": None, "problem": problem}))
else:
print(f"🛡️ security – ({problem}: {HAND})")
return 4
counts = {severity: sum(f["severity"] == severity for f in found) for severity in WEIGHTS}
value = max(0, SAFE - sum(WEIGHTS[s] * n for s, n in counts.items()))
if args.json:
print(json.dumps({"security": value, "files": scanned, **counts, "findings": found}))
else:
places = [f"{f['severity']} {f['path']}:{f['line'] if f['line'] is not None else '?'}" for f in found]
listed = ", ".join(places[:SHOWN]) + (f", +{len(places) - SHOWN} more" if len(places) > SHOWN else "")
detail = f": {listed}" if places else ""
print(f"🛡️ security {value} % (medusa: {plural(len(found), 'finding')} in {plural(scanned, 'file')}{detail})")
if counts["critical"]:
return 1
return 3 if value < SAFE else 0
if __name__ == "__main__":
sys.exit(main())
+17 -11
View File
@@ -19,12 +19,15 @@ description: >
3. Nothing similar found, or the operator picked the staged version: run the
kvb-git-commit-no-verify skill on the currently staged changes
(kvb-code-no-comments audit and kvb-check-legal check first, then the
full staged diff and its score block of 📁 worktree, 🌿 branch,
📊 diff stat, score line and gate line, then `git commit --no-verify`,
exact staging index only). Its approval-prompt duty is binding: the commit
call's approval prompt names worktree, branch, diff stat and the score
values, or the call is not sent. Gate exit 1 is shown and the operator's
word decides; exit 3 asks the operator again first, naming the cause.
full staged diff and the kvb-check-score commit table of 📁 worktree,
🌿 branch, 📊 diff stat, scores with 🛡️ security and gate, then
`git commit --no-verify`, exact staging index only, then the table again
with the hash). Its approval-prompt duty is binding: the commit call's
approval prompt names worktree, branch, diff stat and the score values
with 🛡️ security, or the call is not sent. Gate exit 4 (critical security
finding): no commit, the call is never sent. Exit 1 is shown and the
operator's word decides; exit 3 asks the operator again first, naming
each cause.
4. Group the remaining uncommitted changes into logical bundles (one concern
per bundle: same failure class, mechanism or role) and stage the next one;
never mix concerns to save a commit. Staging happens here and nowhere
@@ -37,8 +40,11 @@ description: >
<worktree> --file <path> --match <regex>`, first with `--dry-run`.
5. For the newly staged bundle run the kvb-think-explain skill (what the
changes do and how the pieces interact, file:line-anchored), then the
kvb-check-confidence skill (its kvb-check-score block, evidence,
residuals).
6. Close with the staged bundle's score block (📁 worktree, 🌿 branch,
📊 diff stat, score line, gate line), what remains unstaged, and wait for
the operator's commit decision; never auto-commit the newly staged bundle.
kvb-check-confidence skill (its kvb-check-score block with 🛡️ security
from `security.py` of kvb-check-security, evidence, residuals).
6. Close with the staged bundle's commit table (`score.py ...
--commit-table <worktree>`: 📁 worktree, 🌿 branch, 📊 diff stat, scores,
gate), what remains unstaged, and wait for the operator's commit
decision; never auto-commit the newly staged bundle. A gate exit 4 says
up front that the bundle cannot be committed until its critical finding
is fixed.
+18 -16
View File
@@ -10,24 +10,26 @@ description: >
then the kvb-check-legal check over the staging index. A legal blocker stops
here.
2. Show in the message right before the `git commit` call, never only after
it, the full staged diff (`git diff --cached`), then the score block:
- 📁 **Worktree**: absolute path from `git rev-parse --show-toplevel`.
- 🌿 **Branch**: `git branch --show-current`, or `detached at <short sha>`.
- 📊 **Diff stat** of what gets committed: `git diff --cached
--shortstat`, e.g. `7 files +163/-108`.
- The score line and gate line of the kvb-check-score skill for the staged
bundle, as the kvb-check-confidence skill produces them: break risk
derived through the kvb-check-break-risk skill, never estimated.
it, the full staged diff (`git diff --cached`), then the commit table of
the kvb-check-score skill (`score.py ... --commit-table <worktree>`):
📁 worktree, 🌿 branch, 📊 staged diff stat, then fix, break, efficiency,
speed, security, score and gate of the staged bundle, as the
kvb-check-confidence skill produces them: break risk derived through the
kvb-check-break-risk skill, never estimated.
3. Approval-prompt duty: the commit call's approval prompt (Claude Code: the
Bash `description`; other runtimes: their prompt text) names the worktree
path, the branch, the diff stat and the score values (fix, break,
efficiency, speed, score). Missing any item: do not send the call. Gate
exit 1 (below 90/5) is shown, the operator's word decides; exit 3 means
ask the operator again before the commit, naming the cause the gate line
prints. The same duty holds for
every call that creates a commit, `git merge` included (there the
incoming diff stat, and the incoming diff in the message before).
path, the branch, the diff stat and the score values (🎯 fix, 💥 break,
🌱 efficiency, ⚡ speed, 🛡️ security, 🏆 score). Missing any item: do not
send the call. Gate exit 4 (critical security finding) means no commit:
the call is never sent, not even on the operator's word; fix and rescan
first. Exit 1 (below 90/5) is shown, the operator's word decides; exit 3
means ask the operator again before the commit, naming each cause the
gate prints. The same duty holds for every call that creates a commit,
`git merge` included (there the incoming diff stat, and the incoming diff
in the message before).
4. Commit exactly the staging index with `git commit --no-verify` and a
Conventional-Commits message derived from the staged diff; never add
unstaged files, never push.
5. Report the created commit hash and what remains uncommitted.
5. Report the created commit hash, the commit table again with
`--commit <hash>` (the committed diff stat and a 🔖 Commit column), and
what remains uncommitted.
+49
View File
@@ -0,0 +1,49 @@
---
name: kvb-think-choice
description: >
Offer the open decisions as multiple choice and checkboxes. Trigger on
"auswahl", "offene punkte".
---
The operator decides faster by picking than by reading prose. This skill
collects every decision still open and offers each as a choice with the
recommended option first.
## Step 1: collect
Gather what waits on the operator, never what an agent can decide itself:
- This session's own open questions and staged bundles waiting for a commit
word.
- `${CLAUDE_SKILL_DIR}/../kvb-org-operator/scripts/operator.py list --json`:
every row with the check `decision`.
- The `kvb-org-rasci` register (`rasci.py show --wide`): tasks `blocked` on
the Operator as A or C.
Merge duplicates and drop items a command can settle (those go to
`operator.py run`). Rank by what unblocks most work first.
## Step 2: shape each item
- One question per item, two to four options, the recommended option first
and marked as such, each with one line on its consequence.
- Independent yes/no items of one kind (which tasks to start, which rows to
close) become one checkbox question.
- An item whose options need a comparison carries a short preview: the
command, the diff stat or the score line of each option.
## Step 3: ask
- **Claude Code**: `AskUserQuestion`, at most four questions per call,
`multiSelect: true` for checkbox questions, `false` otherwise; more items
follow in the next call.
- **Other runtimes**: print the same items as markdown, numbered questions
with lettered options and `- [ ]` boxes, and ask for an answer such as
`1a 2c 3: x, z`.
## Step 4: act
Carry out each answer within its own skill's rules (a commit word through
`kvb-git-commit-next`, a decision row through `operator.py done`). A pick
that needs an action outside the sandbox becomes an `operator.py add` row.
An answer is the operator's word for that item only and never carries over.
+167 -51
View File
@@ -1,103 +1,219 @@
"""Exercise score.py of kvb-check-score: formula, dash and new values, capping, every gate verdict and exit code."""
"""Exercise score.py of kvb-check-score: formula, dash and new values, security, capping, gate verdicts, tables."""
from __future__ import annotations
import os
import shutil
import subprocess
import sys
import tempfile
import unittest
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parents[1]
SCORE = REPO_ROOT / "skills" / "kvb-check-score" / "scripts" / "score.py"
PASS = "✅ gate: 90/5, efficiency and speed <= 100"
PASS = "✅ gate: 90/5, efficiency and speed <= 100, security 100 %"
BASE = {"fix": "95", "break": "2", "efficiency": "100", "speed": "100", "security": "100", "critical": "0"}
HEADER = "| 🎯 Fix | 💥 Break | 🌱 Efficiency | ⚡ Speed | 🛡️ Security | 🏆 Score | Gate |"
ASK = ": ask the operator before the commit"
ENV = {"PATH": os.environ["PATH"]}
def flags(values: dict[str, str]) -> list[str]:
return [item for key, value in values.items() for item in (f"--{key}", value)]
class TestScore(unittest.TestCase):
def _run(self, *args: str) -> subprocess.CompletedProcess:
return subprocess.run([sys.executable, str(SCORE), *args], capture_output=True, text=True, check=False)
def _run(self, *args: str, cwd: Path) -> subprocess.CompletedProcess:
return subprocess.run(
[sys.executable, str(SCORE), *args], cwd=cwd, env=ENV, capture_output=True, text=True, check=False
)
def _score(self, fix: str, risk: str, efficiency: str, speed: str) -> tuple[list[str], int]:
result = self._run("--fix", fix, "--break", risk, "--efficiency", efficiency, "--speed", speed)
def _score(self, **values: str) -> tuple[list[str], int]:
result = self._run(*flags({**BASE, **values}), cwd=REPO_ROOT)
self.assertEqual(result.stderr, "")
return result.stdout.splitlines(), result.returncode
def test_formula_and_line_format(self):
self.assertEqual(
self._score("95", "4", "100", "100"),
(["🎯 fix 95 % · 💥 break 4 % · 🌱 efficiency 100 % · ⚡ speed 100 % · 🏆 score 91", PASS], 0),
self._score(fix="95", **{"break": "4"}),
(["🎯 fix 95 % · 💥 break 4 % · 🌱 efficiency 100 % · ⚡ speed 100 % · 🛡️ security 100 % · 🏆 score 91",
PASS], 0),
)
lines, code = self._score(efficiency="97", speed="-")
self.assertEqual(
lines[0], "🎯 fix 95 % · 💥 break 2 % · 🌱 efficiency 97 % · ⚡ speed – · 🛡️ security 100 % · 🏆 score 96"
)
lines, code = self._score("95", "2", "97", "-")
self.assertEqual(lines[0], "🎯 fix 95 % · 💥 break 2 % · 🌱 efficiency 97 % · ⚡ speed – · 🏆 score 96")
self.assertEqual(code, 0)
self.assertEqual(self._score("92.5", "0", "100", "100")[0][0].split(" · ")[0], "🎯 fix 92.5 %")
self.assertEqual(self._score(fix="92.5")[0][0].split(" · ")[0], "🎯 fix 92.5 %")
def test_dash_values_count_as_100(self):
lines, code = self._score(fix="92", efficiency="-", speed="-", **{"break": "3"})
self.assertEqual(
self._score("92", "3", "-", "-"),
(["🎯 fix 92 % · 💥 break 3 % · 🌱 efficiency – · ⚡ speed – · 🏆 score 89", PASS], 0),
(lines, code),
(["🎯 fix 92 % · 💥 break 3 % · 🌱 efficiency – · ⚡ speed – · 🛡️ security 100 % · 🏆 score 89", PASS], 0),
)
def test_security_multiplies_and_asks_below_100(self):
lines, code = self._score(fix="100", efficiency="-", speed="-", security="75", **{"break": "0"})
self.assertEqual(
(lines, code),
(["🎯 fix 100 % · 💥 break 0 % · 🌱 efficiency – · ⚡ speed – · 🛡️ security 75 % · 🏆 score 75",
f"⚠️ gate: security 75 % < 100{ASK}"], 3),
)
lines, code = self._score(security="~80")
self.assertIn("🛡️ security ~80 % · 🏆 score 74", lines[0])
self.assertEqual((lines[1], code), (f"⚠️ gate: security ~80 % < 100{ASK}", 3))
self.assertEqual(self._score(security="~100")[1], 0)
def test_score_rounds_half_up(self):
for args, total in (
(("94.5", "0", "-", "-"), 95),
(("90", "5", "100", "100"), 86),
(("91", "5", "-", "-"), 86),
(("97", "0", "97", "-"), 100),
for values, total in (
({"fix": "94.5", "break": "0"}, 95),
({"fix": "90", "break": "5"}, 86),
({"fix": "91", "break": "5"}, 86),
({"fix": "97", "break": "0", "efficiency": "97"}, 100),
):
with self.subTest(args=args):
self.assertTrue(self._score(*args)[0][0].endswith(f"🏆 score {total}"))
with self.subTest(values=values):
self.assertTrue(self._score(**values)[0][0].endswith(f"🏆 score {total}"))
def test_score_is_capped(self):
lines, code = self._score("100", "0", "50", "50")
lines, code = self._score(fix="100", efficiency="50", speed="50", **{"break": "0"})
self.assertTrue(lines[0].endswith("🏆 score 100"))
self.assertEqual((lines[1], code), (PASS, 0))
lines, code = self._score("0", "100", "-", "-")
lines, code = self._score(fix="0", **{"break": "100"})
self.assertTrue(lines[0].endswith("🏆 score 0"))
self.assertEqual(code, 1)
def test_boundaries_pass(self):
self.assertEqual(self._score("90", "5", "100", "100")[1], 0)
self.assertEqual(self._score(fix="90", **{"break": "5"})[1], 0)
def test_90_5_failures_exit_1(self):
for args, verdict in (
(("89.5", "1", "-", "-"), "⛔ gate: fix 89.5 % < 90"),
(("95", "6", "-", "-"), "⛔ gate: break 6 % > 5"),
(("80", "10", "-", "-"), "⛔ gate: fix 80 % < 90, break 10 % > 5"),
(("85", "2", "150", "new"), "⛔ gate: fix 85 % < 90"),
for values, verdict in (
({"fix": "89.5", "break": "1"}, "⛔ gate: fix 89.5 % < 90"),
({"break": "6"}, "⛔ gate: break 6 % > 5"),
({"fix": "80", "break": "10"}, "⛔ gate: fix 80 % < 90, break 10 % > 5"),
(
{"fix": "85", "efficiency": "150", "speed": "new", "security": "50"},
f"⛔ gate: fix 85 % < 90; efficiency 150 % > 100, speed new, security 50 % < 100{ASK}",
),
):
with self.subTest(args=args):
lines, code = self._score(*args)
with self.subTest(values=values):
lines, code = self._score(**values)
self.assertEqual((lines[1], code), (verdict, 1))
def test_cost_over_100_asks_the_operator_exit_3(self):
cases = {
("120", "100"): "⚠️ gate: efficiency 120 % > 100: ask the operator before the commit",
("-", "130"): "⚠️ gate: speed 130 % > 100: ask the operator before the commit",
("120", "130"): "⚠️ gate: efficiency 120 % > 100, speed 130 % > 100: ask the operator before the commit",
("100.5", "-"): "⚠️ gate: efficiency 100.5 % > 100: ask the operator before the commit",
("new", "-"): "⚠️ gate: efficiency new: ask the operator before the commit",
}
for (efficiency, speed), verdict in cases.items():
with self.subTest(efficiency=efficiency, speed=speed):
lines, code = self._score("95", "2", efficiency, speed)
def test_costs_ask_the_operator_exit_3(self):
for values, verdict in (
({"efficiency": "120"}, f"⚠️ gate: efficiency 120 % > 100{ASK}"),
({"efficiency": "-", "speed": "130"}, f"⚠️ gate: speed 130 % > 100{ASK}"),
({"efficiency": "100.5", "speed": "-"}, f"⚠️ gate: efficiency 100.5 % > 100{ASK}"),
({"efficiency": "new", "speed": "-"}, f"⚠️ gate: efficiency new{ASK}"),
(
{"efficiency": "120", "speed": "130", "security": "90"},
f"⚠️ gate: efficiency 120 % > 100, speed 130 % > 100, security 90 % < 100{ASK}",
),
):
with self.subTest(values=values):
lines, code = self._score(**values)
self.assertEqual((lines[1], code), (verdict, 3))
def test_critical_finding_wins_exit_4(self):
for values in (
{"critical": "1", "security": "50"},
{"critical": "2", "fix": "80", "break": "10", "efficiency": "150", "security": "0"},
):
with self.subTest(values=values):
lines, code = self._score(**values)
self.assertEqual((lines[1], code), ("⛔ gate: critical security finding: do not commit", 4))
def test_new_counts_as_100_in_the_score(self):
lines, _ = self._score("95", "0", "new", "-")
self.assertEqual(lines[0], "🎯 fix 95 % · 💥 break 0 % · 🌱 efficiency new · ⚡ speed – · 🏆 score 95")
lines, _ = self._score(efficiency="new", speed="-", **{"break": "0"})
self.assertEqual(
lines[0], "🎯 fix 95 % · 💥 break 0 % · 🌱 efficiency new · ⚡ speed – · 🛡️ security 100 % · 🏆 score 95"
)
def test_table(self):
result = self._run(*flags({**BASE, "efficiency": "97", "speed": "-"}), "--table", cwd=REPO_ROOT)
self.assertEqual(
result.stdout.splitlines(),
[
HEADER,
"| --- | --- | --- | --- | --- | --- | --- |",
"| 95 % | 2 % | 97 % | – | 100 % | 96 | ✅ 90/5, efficiency and speed <= 100, security 100 % |",
],
)
self.assertEqual(result.returncode, 0)
result = self._run(*flags({**BASE, "critical": "1"}), "--table", cwd=REPO_ROOT)
self.assertTrue(result.stdout.splitlines()[2].endswith("| ⛔ critical security finding: do not commit |"))
self.assertEqual(result.returncode, 4)
def test_commit_table(self):
tmp = tempfile.TemporaryDirectory()
self.addCleanup(tmp.cleanup)
repo = Path(tmp.name) / "repo"
repo.mkdir()
env = {
"PATH": os.path.dirname(shutil.which("git")),
"HOME": tmp.name,
"GIT_CONFIG_NOSYSTEM": "1",
"GIT_AUTHOR_NAME": "Test",
"GIT_AUTHOR_EMAIL": "test@example.org",
"GIT_COMMITTER_NAME": "Test",
"GIT_COMMITTER_EMAIL": "test@example.org",
}
def git(*args: str) -> str:
return subprocess.run(
["git", *args], cwd=repo, env=env, capture_output=True, text=True, check=True
).stdout.strip()
git("init", "-q", "-b", "main")
(repo / "a.txt").write_text("one\n", encoding="utf-8")
git("add", "a.txt")
git("commit", "-qm", "base")
(repo / "a.txt").write_text("two\n", encoding="utf-8")
(repo / "b.txt").write_text("three\nfour\n", encoding="utf-8")
git("add", "a.txt", "b.txt")
head = ["📁 Worktree", "🌿 Branch", "📊 Diff stat"]
score = "| 95 % | 2 % | 100 % | 100 % | 100 % | 93 | ✅ 90/5, efficiency and speed <= 100, security 100 % |"
result = self._run(*flags(BASE), "--commit-table", str(repo), cwd=repo)
self.assertEqual(result.returncode, 0, result.stderr)
self.assertEqual(
result.stdout.splitlines(),
[
"| " + " | ".join(head) + " " + HEADER,
"|" + " --- |" * 10,
f"| {repo.resolve()} | main | 2 files +3/-1 " + score,
],
)
git("commit", "-qm", "change")
sha = git("rev-parse", "--short", "HEAD")
result = self._run(*flags(BASE), "--commit-table", str(repo), "--commit", "HEAD", cwd=repo)
self.assertEqual(result.stdout.splitlines()[0], "| " + " | ".join([*head, "🔖 Commit"]) + " " + HEADER)
self.assertEqual(result.stdout.splitlines()[2], f"| {repo.resolve()} | main | 2 files +3/-1 | {sha} " + score)
git("checkout", "-q", "--detach", "HEAD")
result = self._run(*flags(BASE), "--commit-table", str(repo), cwd=repo)
self.assertEqual(result.stdout.splitlines()[2], f"| {repo.resolve()} | detached at {sha} | 0 files +0/-0 " + score)
self.assertEqual(self._run(*flags(BASE), "--commit-table", tmp.name, cwd=repo).returncode, 2)
def test_usage_errors_exit_2(self):
good = {"--fix": "95", "--break": "2", "--efficiency": "100", "--speed": "100"}
bad = (
("--fix", "101"), ("--fix", "-1"), ("--fix", "nan"), ("--break", "x"), ("--break", "-"),
("--efficiency", "0"), ("--efficiency", "-5"), ("--efficiency", "abc"), ("--speed", "inf"),
("fix", "101"), ("fix", "-1"), ("fix", "nan"), ("break", "x"), ("break", "-"),
("efficiency", "0"), ("efficiency", "-5"), ("efficiency", "abc"), ("speed", "inf"),
("security", "101"), ("security", "~"), ("security", "~x"), ("security", "-"),
("critical", "-1"), ("critical", "a"), ("critical", "1.5"),
)
for flag, value in bad:
with self.subTest(flag=flag, value=value):
args = [item for key, val in {**good, flag: value}.items() for item in (key, val)]
result = self._run(*args)
for key, value in bad:
with self.subTest(flag=key, value=value):
result = self._run(*flags({**BASE, key: value}), cwd=REPO_ROOT)
self.assertEqual((result.returncode, result.stdout), (2, ""))
self.assertEqual(self._run("--fix", "95", "--break", "2", "--efficiency", "100").returncode, 2)
for missing in ("security", "critical"):
with self.subTest(missing=missing):
values = {k: v for k, v in BASE.items() if k != missing}
self.assertEqual(self._run(*flags(values), cwd=REPO_ROOT).returncode, 2)
for extra in (("--commit", "HEAD"), ("--table", "--commit-table", str(REPO_ROOT))):
with self.subTest(extra=extra):
self.assertEqual(self._run(*flags(BASE), *extra, cwd=REPO_ROOT).returncode, 2)
if __name__ == "__main__":
+152
View File
@@ -0,0 +1,152 @@
"""Exercise security.py of kvb-check-security against a fake medusa: weights, locations, exit codes, hand review."""
from __future__ import annotations
import json
import os
import shutil
import subprocess
import sys
import tempfile
import unittest
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parents[1]
SECURITY = REPO_ROOT / "skills" / "kvb-check-security" / "scripts" / "security.py"
GIT_DIR = os.path.dirname(shutil.which("git"))
PLACEHOLDER = "placeholder-value-never-shown"
FAKE = """#!{python}
import json, os, pathlib, sys
args = sys.argv[1:]
pathlib.Path(__file__).with_name("argv.json").write_text(json.dumps(args))
target, out = pathlib.Path(args[1]), pathlib.Path(args[args.index("-o") + 1])
out.mkdir(parents=True)
findings = []
for path in sorted(p for p in target.rglob("*") if p.is_file()):
for number, text in enumerate(path.read_text().splitlines(), 1):
for severity in ("CRITICAL", "HIGH", "MEDIUM", "LOW", "INFO"):
if "MARK-" + severity in text:
findings.append({{"severity": severity, "file": str(path), "line": number, "code": text}})
report = {{"issues": findings}} if os.environ.get("FAKE_MEDUSA_SHAPE") == "other" else {{"findings": findings}}
(out / "medusa-scan-20261104-120000.json").write_text(json.dumps(report))
"""
HAND = "review trust boundaries, secrets and injection in the diff by hand, then pass --security '~<n>' to kvb-check-score"
class TestSecurity(unittest.TestCase):
def setUp(self):
tmp = tempfile.TemporaryDirectory()
self.addCleanup(tmp.cleanup)
self.base = Path(tmp.name)
self.bin = self.base / "bin"
self.bin.mkdir()
medusa = self.bin / "medusa"
medusa.write_text(FAKE.format(python=sys.executable), encoding="utf-8")
medusa.chmod(0o755)
self.repo = self.base / "repo"
self.repo.mkdir()
self.env = {
"PATH": f"{self.bin}:{GIT_DIR}",
"HOME": str(self.base),
"GIT_CONFIG_NOSYSTEM": "1",
"GIT_AUTHOR_NAME": "Test",
"GIT_AUTHOR_EMAIL": "test@example.org",
"GIT_COMMITTER_NAME": "Test",
"GIT_COMMITTER_EMAIL": "test@example.org",
}
self.git("init", "-q", "-b", "main")
self.write("app.py", "print(1)\n")
self.write("old.py", "x = 1 # MARK-CRITICAL\n")
self.write("gone.py", "y = 2 # MARK-HIGH\n")
self.git("add", "-A")
self.git("commit", "-qm", "base")
def git(self, *args: str) -> None:
subprocess.run(["git", *args], cwd=self.repo, env=self.env, capture_output=True, check=True)
def write(self, path: str, text: str) -> None:
(self.repo / path).write_text(text, encoding="utf-8")
def scan(self, *extra: str, env: dict[str, str]) -> subprocess.CompletedProcess:
return subprocess.run(
[sys.executable, str(SECURITY), "--repo", str(self.repo), "--base", "HEAD", *extra],
cwd=self.base, env=env, capture_output=True, text=True, check=False,
)
def test_changed_and_untracked_files_are_weighted(self):
self.write("app.py", f"print(1)\ntoken = '{PLACEHOLDER}' # MARK-HIGH\n")
self.write("new.py", "a = 1 # MARK-LOW\nb = 2\nc = 3 # MARK-MEDIUM\nd = 4 # MARK-INFO\n")
(self.repo / "gone.py").unlink()
(self.repo / "link.py").symlink_to(self.repo / "old.py")
result = self.scan(env=self.env)
self.assertEqual(
result.stdout,
"🛡️ security 63 % (medusa: 4 findings in 2 files: high app.py:2, medium new.py:3, low new.py:1, "
"info new.py:4)\n",
)
self.assertEqual(result.returncode, 3, result.stderr)
argv = json.loads((self.bin / "argv.json").read_text(encoding="utf-8"))
self.assertEqual([argv[0], *argv[2:4], argv[4], argv[6]], ["scan", "--format", "json", "-o", "--no-cache"])
result = self.scan("--json", env=self.env)
self.assertNotIn(PLACEHOLDER, result.stdout + result.stderr)
found = json.loads(result.stdout)
self.assertEqual(
{k: found[k] for k in ("security", "files", "critical", "high", "medium", "low", "info")},
{"security": 63, "files": 2, "critical": 0, "high": 1, "medium": 1, "low": 1, "info": 1},
)
self.assertEqual(found["findings"][0], {"severity": "high", "path": "app.py", "line": 2})
def test_critical_exits_1_and_floors_at_0(self):
self.write("app.py", "a # MARK-CRITICAL\nb # MARK-CRITICAL\nc # MARK-CRITICAL\n")
result = self.scan(env=self.env)
self.assertTrue(result.stdout.startswith("🛡️ security 0 % (medusa: 3 findings in 1 file: critical app.py:1, "))
self.assertEqual(result.returncode, 1)
def test_more_than_five_locations_are_cut(self):
self.write("app.py", "".join(f"v{n} # MARK-LOW\n" for n in range(7)))
result = self.scan(env=self.env)
self.assertEqual(
result.stdout,
"🛡️ security 86 % (medusa: 7 findings in 1 file: low app.py:1, low app.py:2, low app.py:3, "
"low app.py:4, low app.py:5, +2 more)\n",
)
self.assertEqual(result.returncode, 3)
def test_clean_and_empty_changes_are_100(self):
result = self.scan(env=self.env)
self.assertEqual((result.stdout, result.returncode), ("🛡️ security 100 % (medusa: 0 findings in 0 files)\n", 0))
self.assertFalse((self.bin / "argv.json").exists())
self.write("app.py", "print(2)\n")
result = self.scan(env=self.env)
self.assertEqual((result.stdout, result.returncode), ("🛡️ security 100 % (medusa: 0 findings in 1 file)\n", 0))
def test_missing_medusa_asks_for_a_review_by_hand(self):
if shutil.which("medusa", path=GIT_DIR):
self.skipTest("a real medusa sits next to git")
self.write("app.py", "print(2)\n")
result = self.scan(env={**self.env, "PATH": GIT_DIR})
self.assertEqual((result.stdout, result.returncode), (f"🛡️ security – (medusa missing: {HAND})\n", 4))
result = self.scan("--json", env={**self.env, "PATH": GIT_DIR})
self.assertEqual(json.loads(result.stdout), {"security": None, "problem": "medusa missing"})
def test_unreadable_report_asks_for_a_review_by_hand(self):
self.write("app.py", "print(2)\n")
result = self.scan(env={**self.env, "FAKE_MEDUSA_SHAPE": "other"})
self.assertEqual(result.returncode, 4)
self.assertRegex(result.stdout, r"^🛡️ security – \(medusa report unreadable: .*holds no findings list")
def test_usage_errors_exit_2(self):
result = subprocess.run(
[sys.executable, str(SECURITY), "--repo", str(self.repo), "--base", "nope"],
env=self.env, capture_output=True, text=True, check=False,
)
self.assertEqual(result.returncode, 2)
result = subprocess.run(
[sys.executable, str(SECURITY), "--repo", str(self.base / "absent"), "--base", "HEAD"],
env=self.env, capture_output=True, text=True, check=False,
)
self.assertEqual(result.returncode, 2)
if __name__ == "__main__":
unittest.main()