fix(kvb-check-security): give medusa a configurable scanner timeout and fail closed
medusa-security caps every scanner at a fixed 60 s per file, so long files lost scanner results while the run still scored 100 %. security.py now passes --timeout (MEDUSA_SCANNER_TIMEOUT) and an overall --deadline to the medusa run (1 to 86400 s), runs medusa in its own process group that dies with security.py (stop signals, exceptions, PR_SET_PDEATHSIG), keeps signals the caller ignores ignored, and ends an incomplete or killed scan with exit 4, nothing cached and nothing scored. The fault text names only causes medusa can tell apart, and the cache key holds the timeout medusa actually got. 🎯 Fix: 92 % 💥 Break: 4 % 💸 Tokens: 100 % ⏱️ Velocity: 100 % 🛡️ Security: 100 % 🏆 Score: 88 🚦 Gate: ✅ 90/5, tokens and velocity <= 100, security 100 % Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
a2a03995cb
commit
f145a600fe
4 files changed
+348
-39
No files matched your search
@@ -10,7 +10,7 @@ MEDUSA is a defensive AI-first security scanner (40,000+ patterns: CVEs, prompt
|
|||||||
## Setup
|
## Setup
|
||||||
|
|
||||||
Check availability with `medusa --version`.
|
Check availability with `medusa --version`.
|
||||||
The skills repository's `make install` installs it with pipx; if it is still missing, `pipx install medusa-security` is the operator's step.
|
The skills repo's `make install` installs it with pipx; else `pipx install medusa-security` is the operator's step.
|
||||||
|
|
||||||
## Commands
|
## Commands
|
||||||
|
|
||||||
@@ -32,11 +32,12 @@ python3 ${CLAUDE_SKILL_DIR}/scripts/security.py --repo <root> --base <ref>
|
|||||||
It scans the files changed since `<ref>`, untracked ones included, once and
|
It scans the files changed since `<ref>`, untracked ones included, once and
|
||||||
prints one line: 100 % is safe, each critical, high, medium and low finding
|
prints one line: 100 % is safe, each critical, high, medium and low finding
|
||||||
costs 50, 25, 10 and 2. Pass the value as `--security` and the critical count
|
costs 50, 25, 10 and 2. Pass the value as `--security` and the critical count
|
||||||
(`--json`) as `--critical` to `score.py`. A complete run is cached.
|
(`--json`) as `--critical` to `score.py`. A complete run is cached. Run long
|
||||||
|
files in the background, the log under `skills/.state`.
|
||||||
|
|
||||||
- Exit 0: 100 %. Exit 3: below 100 without a critical; the score gate asks
|
- Exit 0: 100 %. Exit 3: below 100 without a critical; the score gate asks
|
||||||
the operator. Exit 1: a critical; the gate forbids the commit. Exit 2: usage.
|
the operator. Exit 1: a critical; the gate forbids the commit. Exit 2: usage.
|
||||||
- A `not covered` list or exit 4: read `coverage.md` next to this file first.
|
- `not covered` or exit 4: read `coverage.md` first.
|
||||||
- **Accepted false positives**: `security.py accept --repo <root> --rule <id>
|
- **Accepted false positives**: `security.py accept --repo <root> --rule <id>
|
||||||
[--rule <id> ...] --path <path> --sha1 <sha1> --line <n> --reason "<one line>"`
|
[--rule <id> ...] --path <path> --sha1 <sha1> --line <n> --reason "<one line>"`
|
||||||
(values from `--json`) keeps a finding out of the score until its line or
|
(values from `--json`) keeps a finding out of the score until its line or
|
||||||
@@ -46,7 +47,7 @@ costs 50, 25, 10 and 2. Pass the value as `--security` and the critical count
|
|||||||
|
|
||||||
## Rules
|
## Rules
|
||||||
|
|
||||||
- Run scans read-only; never pass `--reveal` and never run `medusa secrets purge` unless the user explicitly asks for it.
|
- Run scans read-only; never pass `--reveal` and never run `medusa secrets purge` unless the user asks.
|
||||||
- Summarize findings by severity with file/line references; do not paste secret values into the conversation, even redacted ones.
|
- Summarize findings by severity with file/line references; never paste secret values, even redacted ones.
|
||||||
- Treat findings as signals for review, not verdicts; confirm true positives in the code before proposing fixes.
|
- Treat findings as signals for review, not verdicts; confirm true positives in the code before fixing.
|
||||||
- Vet third-party agent repos (`medusa scan --git`) before recommending the user clone or install them.
|
- Vet third-party agent repos (`medusa scan --git`) before recommending a clone or install.
|
||||||
@@ -1,14 +1,15 @@
|
|||||||
# Coverage of a security.py scan
|
# Coverage of a security.py scan
|
||||||
|
|
||||||
The score counts only files medusa really scans; its types, path exclusions
|
The score counts only files medusa scans; its types, exclusions and limits
|
||||||
and limits come from medusa's own source.
|
come from its source.
|
||||||
|
|
||||||
- `not covered`: a type medusa never scans (`*.j2`, `Makefile`) or a path its
|
- `not covered`: a type medusa never scans (`*.j2`, `Makefile`) or a path its
|
||||||
default exclusions drop (any path holding `env`, `build`, `dist`, `obj`,
|
default exclusions drop (`env`, `build`, `dist`, `obj`, `vendor` and the
|
||||||
`vendor` and the like). Review those files by hand.
|
like). Review them by hand.
|
||||||
- `run incomplete`: mostly a fully loaded CPU, which pushes scanners past
|
- `run incomplete`: a scanner passed `--timeout` or the run `--deadline`,
|
||||||
medusa's per-file timeout so their findings vanish.
|
mostly on a loaded CPU or a long file. A plain `PER_SCANNER_TIMEOUT` number
|
||||||
- `never covers`: a rerun does not help; the named file stays over medusa's
|
counts as ignoring `MEDUSA_SCANNER_TIMEOUT`, even if read elsewhere.
|
||||||
limit or out of its scan.
|
- `never covers`: a rerun does not help; the file stays over medusa's limit
|
||||||
|
or out of its scan.
|
||||||
- On exit 4 keep the quotes of the hand value (zsh expands a bare `~100`) and
|
- On exit 4 keep the quotes of the hand value (zsh expands a bare `~100`) and
|
||||||
pass the criticals you found as `--critical`.
|
pass the criticals you found as `--critical`.
|
||||||
@@ -5,6 +5,13 @@ Args:
|
|||||||
--repo: git repository whose working tree holds the change.
|
--repo: git repository whose working tree holds the change.
|
||||||
--base: git ref the change starts from.
|
--base: git ref the change starts from.
|
||||||
--json: print the figures as JSON instead of the line.
|
--json: print the figures as JSON instead of the line.
|
||||||
|
--timeout: whole seconds, 1 to CEILING, each medusa scanner may spend on
|
||||||
|
one file, handed to medusa as KNOB in place of any the caller set;
|
||||||
|
default TIMEOUT.
|
||||||
|
--deadline: whole seconds, 1 to CEILING, the whole medusa run may take;
|
||||||
|
past it its process group is killed and the run is incomplete;
|
||||||
|
default DEADLINE. A STOPS signal the caller does not ignore kills
|
||||||
|
that group too, then this process by the same signal.
|
||||||
|
|
||||||
accept --repo <root> --rule <rule_id> [--rule ...] --path <repo path> --sha1 <40 hex> [--line <n>] --reason <one line>:
|
accept --repo <root> --rule <rule_id> [--rule ...] --path <repo path> --sha1 <40 hex> [--line <n>] --reason <one line>:
|
||||||
record reviewed false positives of one line instead, a row per rule; --json
|
record reviewed false positives of one line instead, a row per rule; --json
|
||||||
@@ -17,7 +24,8 @@ Scan:
|
|||||||
out, copied to a temporary tree and scanned once with
|
out, copied to a temporary tree and scanned once with
|
||||||
`medusa scan <tree> --format json -o <dir> --no-cache --workers <w>`, w the
|
`medusa scan <tree> --format json -o <dir> --no-cache --workers <w>`, w the
|
||||||
files scanned but at most WORKERS, HOME pointing into the temporary
|
files scanned but at most WORKERS, HOME pointing into the temporary
|
||||||
directory because medusa creates ~/.medusa on start, read from
|
directory because medusa creates ~/.medusa on start and KNOB set to
|
||||||
|
--timeout (a medusa that ignores KNOB keeps its own cap), read from
|
||||||
`<dir>/medusa-scan-*.json`: a `findings` list whose entries carry
|
`<dir>/medusa-scan-*.json`: a `findings` list whose entries carry
|
||||||
`severity` (critical, high, medium, low or info), `rule_id`, `file` and `line`.
|
`severity` (critical, high, medium, low or info), `rule_id`, `file` and `line`.
|
||||||
medusa skips the SKIPPED suffixes (.mjs, .cjs, any case), so such a file is
|
medusa skips the SKIPPED suffixes (.mjs, .cjs, any case), so such a file is
|
||||||
@@ -48,15 +56,19 @@ Scan:
|
|||||||
`files_scanned` and, as medusa counts each file's LF bytes plus one and a
|
`files_scanned` and, as medusa counts each file's LF bytes plus one and a
|
||||||
skipped file as 0, their sum in `lines_scanned`. The run is complete only when medusa also exits 0 and its
|
skipped file as 0, their sum in `lines_scanned`. The run is complete only when medusa also exits 0 and its
|
||||||
output names no FAULT (a scanner error or a timeout: a scanner over its
|
output names no FAULT (a scanner error or a timeout: a scanner over its
|
||||||
per-file timeout, as under a fully loaded CPU, adds nothing to the report;
|
per-file timeout, as on a long file or under a fully loaded CPU, adds
|
||||||
|
nothing to the report;
|
||||||
a directory left unread, both printed by medusa's main process) and no
|
a directory left unread, both printed by medusa's main process) and no
|
||||||
SKIP. A SKIP line is a second signal only: medusa runs with `python -E`,
|
SKIP. A SKIP line is a second signal only: medusa runs with `python -E`,
|
||||||
so its pool workers print it into a buffer that dies with them.
|
so its pool workers print it into a buffer that dies with them. A TIMED
|
||||||
|
FAULT names RERUN, unless core/parallel.py assigns PER_SCANNER_TIMEOUT
|
||||||
|
a plain number: medusa then ignores KNOB, and the problem names CAPPED
|
||||||
|
with that number when the run lasted at least as long, else ERRED.
|
||||||
|
|
||||||
Cache:
|
Cache:
|
||||||
The findings of a complete scan, keyed by the sha256 of this script, the
|
The findings of a complete scan, keyed by the sha256 of this script, the
|
||||||
resolved medusa binary with its mtime and size, PATH, the files not covered,
|
resolved medusa binary with its mtime and size, PATH, the files not covered,
|
||||||
every MEDUSA_* variable and the tree path, repository path, mode and
|
every MEDUSA_* variable handed to medusa and the tree path, repository path, mode and
|
||||||
content sha256 of every copied file, in
|
content sha256 of every copied file, in
|
||||||
<council root>/security-cache/<key>.json for CACHE_SECONDS. A later scan
|
<council root>/security-cache/<key>.json for CACHE_SECONDS. A later scan
|
||||||
with the same key reads them instead of running medusa; any edit of a
|
with the same key reads them instead of running medusa; any edit of a
|
||||||
@@ -90,7 +102,7 @@ Output:
|
|||||||
Exit codes:
|
Exit codes:
|
||||||
0 security 100, 1 a critical finding, 2 usage error, 3 under 100 without a
|
0 security 100, 1 a critical finding, 2 usage error, 3 under 100 without a
|
||||||
critical finding, 4 medusa missing, its report or RULES unreadable, its
|
critical finding, 4 medusa missing, its report or RULES unreadable, its
|
||||||
run incomplete (rerun once the host is idle), a scanned file it never
|
run incomplete (RERUN, CAPPED, ERRED or past --deadline), a scanned file it never
|
||||||
covers or a diff of files not covered only: review the diff by hand.
|
covers or a diff of files not covered only: review the diff by hand.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
@@ -98,20 +110,25 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import argparse
|
import argparse
|
||||||
import ast
|
import ast
|
||||||
|
import contextlib
|
||||||
|
import ctypes
|
||||||
import datetime
|
import datetime
|
||||||
import fcntl
|
import fcntl
|
||||||
|
import functools
|
||||||
import hashlib
|
import hashlib
|
||||||
import importlib.util
|
import importlib.util
|
||||||
import json
|
import json
|
||||||
import os
|
import os
|
||||||
import re
|
import re
|
||||||
import shutil
|
import shutil
|
||||||
|
import signal
|
||||||
import stat
|
import stat
|
||||||
import subprocess
|
import subprocess
|
||||||
import sys
|
import sys
|
||||||
import tempfile
|
import tempfile
|
||||||
import time
|
import time
|
||||||
from collections import Counter
|
from collections import Counter
|
||||||
|
from collections.abc import Iterable
|
||||||
from fnmatch import fnmatchcase
|
from fnmatch import fnmatchcase
|
||||||
from pathlib import Path, PurePosixPath
|
from pathlib import Path, PurePosixPath
|
||||||
from typing import NamedTuple
|
from typing import NamedTuple
|
||||||
@@ -121,6 +138,10 @@ SAFE = 100
|
|||||||
SHOWN = 5
|
SHOWN = 5
|
||||||
REACH = 2
|
REACH = 2
|
||||||
WORKERS = 4
|
WORKERS = 4
|
||||||
|
KNOB = "MEDUSA_SCANNER_TIMEOUT"
|
||||||
|
TIMEOUT = 900
|
||||||
|
DEADLINE = 3600
|
||||||
|
CEILING = 86400
|
||||||
SKIPPED = (".mjs", ".cjs")
|
SKIPPED = (".mjs", ".cjs")
|
||||||
ALIAS = ".js"
|
ALIAS = ".js"
|
||||||
NAME_MAX = 255
|
NAME_MAX = 255
|
||||||
@@ -132,7 +153,16 @@ SHAPE = {
|
|||||||
"sha1": (str, NOTHING), "context": (str, NOTHING),
|
"sha1": (str, NOTHING), "context": (str, NOTHING),
|
||||||
}
|
}
|
||||||
SHA1 = re.compile(r"[0-9a-f]{40}")
|
SHA1 = re.compile(r"[0-9a-f]{40}")
|
||||||
FAULT = re.compile(r"scanner error\(s\)/timeout\(s\) during scan|could not be read and were skipped")
|
TIMED = re.compile(r"scanner error\(s\)/timeout\(s\) during scan")
|
||||||
|
FAULT = re.compile(rf"{TIMED.pattern}|could not be read and were skipped")
|
||||||
|
RERUN = "rerun once the host is idle"
|
||||||
|
CAPPED = (
|
||||||
|
"a scanner crashed or hit medusa's fixed {cap} s cap (medusa counts both together): rerun when idle; "
|
||||||
|
f"if it repeats, give medusa the {KNOB} knob"
|
||||||
|
)
|
||||||
|
ERRED = "scanner error (no scanner reached the {cap} s cap): rerun when idle"
|
||||||
|
STOPS = (signal.SIGINT, signal.SIGTERM, signal.SIGHUP)
|
||||||
|
PR_SET_PDEATHSIG = 1
|
||||||
SKIP = re.compile(r"Skipped .+ exceeds .+ limit")
|
SKIP = re.compile(r"Skipped .+ exceeds .+ limit")
|
||||||
RULES = {
|
RULES = {
|
||||||
"core/parallel.py": {
|
"core/parallel.py": {
|
||||||
@@ -167,6 +197,10 @@ class Uncovered(Exception):
|
|||||||
pass
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
class Stopped(BaseException):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
class Rules(NamedTuple):
|
class Rules(NamedTuple):
|
||||||
extensions: frozenset
|
extensions: frozenset
|
||||||
names: frozenset
|
names: frozenset
|
||||||
@@ -470,6 +504,86 @@ def rules(folder: Path | None) -> Rules | None:
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def capped(folder: Path) -> int | None:
|
||||||
|
"""Returns:
|
||||||
|
the plain number medusa's core/parallel.py assigns PER_SCANNER_TIMEOUT, its
|
||||||
|
fixed cap that ignores KNOB, else None.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
return assigned(folder / "core" / "parallel.py", {"PER_SCANNER_TIMEOUT": int})["PER_SCANNER_TIMEOUT"]
|
||||||
|
except (OSError, SyntaxError, ValueError, TypeError):
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def advice(short: str, cap: int | None, lasted: float) -> str:
|
||||||
|
"""Returns:
|
||||||
|
what to do about the FAULT or shortfall short of a run that lasted these
|
||||||
|
seconds under medusa's fixed cap: CAPPED, ERRED or RERUN.
|
||||||
|
"""
|
||||||
|
if cap is None or not TIMED.search(short):
|
||||||
|
return RERUN
|
||||||
|
return (CAPPED if lasted >= cap else ERRED).format(cap=cap)
|
||||||
|
|
||||||
|
|
||||||
|
def halt(group: list[int], handled: Iterable[int]) -> None:
|
||||||
|
"""Args:
|
||||||
|
group: process group ids to kill with SIGKILL.
|
||||||
|
handled: signals set to SIG_IGN before the kill.
|
||||||
|
"""
|
||||||
|
for number in handled:
|
||||||
|
signal.signal(number, signal.SIG_IGN)
|
||||||
|
for pid in group:
|
||||||
|
with contextlib.suppress(ProcessLookupError):
|
||||||
|
os.killpg(pid, signal.SIGKILL)
|
||||||
|
|
||||||
|
|
||||||
|
def deathsig() -> functools.partial | None:
|
||||||
|
"""Returns:
|
||||||
|
the preexec_fn that kills medusa when this process dies, None for a libc without prctl.
|
||||||
|
"""
|
||||||
|
prctl = getattr(ctypes.CDLL(None), "prctl", None)
|
||||||
|
return functools.partial(prctl, PR_SET_PDEATHSIG, int(signal.SIGKILL)) if prctl else None
|
||||||
|
|
||||||
|
|
||||||
|
def medusa_run(command: list[str], work: Path, env: dict[str, str], deadline: int) -> tuple[int, str, float]:
|
||||||
|
"""Returns:
|
||||||
|
medusa's exit code, stdout and seconds. medusa runs in its own process
|
||||||
|
group, dies with this process (deathsig) and its group is killed past
|
||||||
|
deadline seconds (Incomplete), on a STOPS signal the caller does not
|
||||||
|
ignore (Stopped) or on any other exception.
|
||||||
|
"""
|
||||||
|
group: list[int] = []
|
||||||
|
previous = {number: signal.getsignal(number) for number in STOPS}
|
||||||
|
previous = {number: handler for number, handler in previous.items() if handler not in (signal.SIG_IGN, None)}
|
||||||
|
|
||||||
|
def stop(number: int, frame: object) -> None:
|
||||||
|
halt(group, previous)
|
||||||
|
raise Stopped(number)
|
||||||
|
|
||||||
|
for number in previous:
|
||||||
|
signal.signal(number, stop)
|
||||||
|
start = time.monotonic()
|
||||||
|
try:
|
||||||
|
with subprocess.Popen(
|
||||||
|
command, cwd=work, stdin=subprocess.DEVNULL, stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, env=env,
|
||||||
|
start_new_session=True, preexec_fn=deathsig(), # noqa: PLW1509
|
||||||
|
) as process:
|
||||||
|
group.append(process.pid)
|
||||||
|
try:
|
||||||
|
out, _ = process.communicate(timeout=deadline)
|
||||||
|
except BaseException:
|
||||||
|
halt(group, previous)
|
||||||
|
raise
|
||||||
|
except subprocess.TimeoutExpired:
|
||||||
|
raise Incomplete(
|
||||||
|
f"medusa ran past its {deadline} s --deadline: rerun it in the background with --deadline raised, up to {CEILING}"
|
||||||
|
) from None
|
||||||
|
finally:
|
||||||
|
for number, handler in previous.items():
|
||||||
|
signal.signal(number, handler)
|
||||||
|
return process.returncode, out.decode(errors="replace"), time.monotonic() - start
|
||||||
|
|
||||||
|
|
||||||
def taken(name: str, found: Rules) -> bool:
|
def taken(name: str, found: Rules) -> bool:
|
||||||
"""Returns:
|
"""Returns:
|
||||||
whether medusa's find_scannable_files takes up a file of this tree path:
|
whether medusa's find_scannable_files takes up a file of this tree path:
|
||||||
@@ -546,13 +660,13 @@ def cut(code: int, said: list[str], summary: dict, sizes: dict[str, tuple[int, i
|
|||||||
return None
|
return None
|
||||||
|
|
||||||
|
|
||||||
def keyed(medusa: str, tree: Path, names: dict[str, str], missed: list[str]) -> str:
|
def keyed(medusa: str, tree: Path, names: dict[str, str], missed: list[str], env: dict[str, str]) -> str:
|
||||||
binary = Path(medusa).resolve()
|
binary = Path(medusa).resolve()
|
||||||
info = binary.stat()
|
info = binary.stat()
|
||||||
key = hashlib.sha256(Path(__file__).read_bytes())
|
key = hashlib.sha256(Path(__file__).read_bytes())
|
||||||
key.update(f"\0{binary}\0{info.st_mtime_ns}\0{info.st_size}\0{os.environ.get('PATH', '')}\0".encode())
|
key.update(f"\0{binary}\0{info.st_mtime_ns}\0{info.st_size}\0{os.environ.get('PATH', '')}\0".encode())
|
||||||
key.update("".join(f"{name}\0" for name in missed).encode() + b"\0")
|
key.update("".join(f"{name}\0" for name in missed).encode() + b"\0")
|
||||||
key.update("".join(f"{k}={v}\0" for k, v in sorted(os.environ.items()) if k.startswith("MEDUSA_")).encode())
|
key.update("".join(f"{k}={v}\0" for k, v in sorted(env.items()) if k.startswith("MEDUSA_")).encode())
|
||||||
for scratch, name in sorted(names.items()):
|
for scratch, name in sorted(names.items()):
|
||||||
path = tree / scratch
|
path = tree / scratch
|
||||||
key.update(f"{scratch}\0{name}\0{path.stat().st_mode & 0o777:o}\0".encode())
|
key.update(f"{scratch}\0{name}\0{path.stat().st_mode & 0o777:o}\0".encode())
|
||||||
@@ -610,7 +724,9 @@ def remember(folder: Path, key: str, found: list[dict]) -> None:
|
|||||||
Path(name).unlink(missing_ok=True)
|
Path(name).unlink(missing_ok=True)
|
||||||
|
|
||||||
|
|
||||||
def scan(medusa: str, root: Path, base: str, cache: Path | None) -> tuple[int, list[dict], list[str]]:
|
def scan(
|
||||||
|
medusa: str, root: Path, base: str, cache: Path | None, timeout: int, deadline: int,
|
||||||
|
) -> tuple[int, list[dict], list[str]]:
|
||||||
"""Returns:
|
"""Returns:
|
||||||
the count of scanned files, their findings and the repository paths
|
the count of scanned files, their findings and the repository paths
|
||||||
of the copied files medusa never takes up.
|
of the copied files medusa never takes up.
|
||||||
@@ -623,8 +739,9 @@ def scan(medusa: str, root: Path, base: str, cache: Path | None) -> tuple[int, l
|
|||||||
names = copy(root, changed(root, base), tree)
|
names = copy(root, changed(root, base), tree)
|
||||||
if not names:
|
if not names:
|
||||||
return 0, [], []
|
return 0, [], []
|
||||||
env = {**os.environ, "HOME": str(home)}
|
env = {**os.environ, "HOME": str(home), KNOB: str(timeout)}
|
||||||
known = rules(package(medusa, work, env))
|
folder = package(medusa, work, env)
|
||||||
|
known = rules(folder)
|
||||||
if known is None:
|
if known is None:
|
||||||
raise Uncovered("medusa's own types, exclusions and limits are unreadable from its source")
|
raise Uncovered("medusa's own types, exclusions and limits are unreadable from its source")
|
||||||
sizes = {names[scratch]: size for scratch, size in sized(tree).items() if taken(scratch, known)}
|
sizes = {names[scratch]: size for scratch, size in sized(tree).items() if taken(scratch, known)}
|
||||||
@@ -635,26 +752,26 @@ def scan(medusa: str, root: Path, base: str, cache: Path | None) -> tuple[int, l
|
|||||||
over = oversized(sizes, known)
|
over = oversized(sizes, known)
|
||||||
if over:
|
if over:
|
||||||
raise Uncovered(over)
|
raise Uncovered(over)
|
||||||
key = keyed(medusa, tree, names, missed)
|
key = keyed(medusa, tree, names, missed, env)
|
||||||
hit = recalled(cache, key) if cache else None
|
hit = recalled(cache, key) if cache else None
|
||||||
if hit is not None:
|
if hit is not None:
|
||||||
return scanned, hit, missed
|
return scanned, hit, missed
|
||||||
result = subprocess.run(
|
code, out, lasted = medusa_run(
|
||||||
[medusa, "scan", str(tree), "--format", "json", "-o", str(reports), "--no-cache",
|
[medusa, "scan", str(tree), "--format", "json", "-o", str(reports), "--no-cache",
|
||||||
"--workers", str(min(scanned, WORKERS))],
|
"--workers", str(min(scanned, WORKERS))],
|
||||||
cwd=work, stdin=subprocess.DEVNULL, capture_output=True, check=False, env=env,
|
work, env, deadline,
|
||||||
)
|
)
|
||||||
try:
|
try:
|
||||||
found, summary = findings(reports, tree, names)
|
found, summary = findings(reports, tree, names)
|
||||||
except Unreadable as error:
|
except Unreadable as error:
|
||||||
raise Unreadable(f"{error} (medusa exit {result.returncode})") from None
|
raise Unreadable(f"{error} (medusa exit {code})") from None
|
||||||
said = [line.replace(f"{tree}/", "").strip(" \t!⚠️") for line in result.stdout.decode(errors="replace").splitlines()]
|
said = [line.replace(f"{tree}/", "").strip(" \t!⚠️") for line in out.splitlines()]
|
||||||
never = uncovered(said, summary, sizes)
|
never = uncovered(said, summary, sizes)
|
||||||
if never:
|
if never:
|
||||||
raise Uncovered(never)
|
raise Uncovered(never)
|
||||||
short = cut(result.returncode, said, summary, sizes)
|
short = cut(code, said, summary, sizes)
|
||||||
if short:
|
if short:
|
||||||
raise Incomplete(short)
|
raise Incomplete(f"{advice(short, capped(folder), lasted)}: {short}")
|
||||||
if cache:
|
if cache:
|
||||||
remember(cache, key, found)
|
remember(cache, key, found)
|
||||||
return scanned, found, missed
|
return scanned, found, missed
|
||||||
@@ -668,11 +785,19 @@ def shown(items: list[str]) -> str:
|
|||||||
return ", ".join(items[:SHOWN]) + (f", +{len(items) - SHOWN} more" if len(items) > SHOWN else "")
|
return ", ".join(items[:SHOWN]) + (f", +{len(items) - SHOWN} more" if len(items) > SHOWN else "")
|
||||||
|
|
||||||
|
|
||||||
|
def seconds(text: str) -> int:
|
||||||
|
if not text.isdecimal() or not 1 <= int(text) <= CEILING:
|
||||||
|
raise argparse.ArgumentTypeError(f"want whole seconds from 1 to {CEILING}, got {text!r}")
|
||||||
|
return int(text)
|
||||||
|
|
||||||
|
|
||||||
def main() -> int:
|
def main() -> int:
|
||||||
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
|
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
|
||||||
parser.add_argument("--repo", required=True)
|
parser.add_argument("--repo", required=True)
|
||||||
parser.add_argument("--base", required=True)
|
parser.add_argument("--base", required=True)
|
||||||
parser.add_argument("--json", action="store_true")
|
parser.add_argument("--json", action="store_true")
|
||||||
|
parser.add_argument("--timeout", type=seconds, default=TIMEOUT)
|
||||||
|
parser.add_argument("--deadline", type=seconds, default=DEADLINE)
|
||||||
args = parser.parse_args()
|
args = parser.parse_args()
|
||||||
|
|
||||||
repo = Path(args.repo)
|
repo = Path(args.repo)
|
||||||
@@ -690,11 +815,11 @@ def main() -> int:
|
|||||||
problem = None if medusa else "medusa missing"
|
problem = None if medusa else "medusa missing"
|
||||||
if medusa:
|
if medusa:
|
||||||
try:
|
try:
|
||||||
scanned, every, missed = scan(medusa, root, args.base, shelf(council.ROOT))
|
scanned, every, missed = scan(medusa, root, args.base, shelf(council.ROOT), args.timeout, args.deadline)
|
||||||
except Unreadable as error:
|
except Unreadable as error:
|
||||||
problem = f"medusa report unreadable: {error}"
|
problem = f"medusa report unreadable: {error}"
|
||||||
except Incomplete as error:
|
except Incomplete as error:
|
||||||
problem = f"medusa run incomplete, nothing cached, rerun once the host is idle: {error}"
|
problem = f"medusa run incomplete, nothing cached, {error}"
|
||||||
except Uncovered as error:
|
except Uncovered as error:
|
||||||
problem = f"medusa never covers this diff, nothing cached: {error}"
|
problem = f"medusa never covers this diff, nothing cached: {error}"
|
||||||
if problem:
|
if problem:
|
||||||
@@ -728,4 +853,8 @@ def main() -> int:
|
|||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
sys.exit(accept(sys.argv[2:]) if sys.argv[1:2] == ["accept"] else main())
|
try:
|
||||||
|
sys.exit(accept(sys.argv[2:]) if sys.argv[1:2] == ["accept"] else main())
|
||||||
|
except Stopped as stopped:
|
||||||
|
signal.signal(stopped.args[0], signal.SIG_DFL)
|
||||||
|
os.kill(os.getpid(), stopped.args[0])
|
||||||
+181
-3
@@ -3,12 +3,15 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import hashlib
|
import hashlib
|
||||||
|
import importlib.util
|
||||||
import json
|
import json
|
||||||
import os
|
import os
|
||||||
import shutil
|
import shutil
|
||||||
|
import signal
|
||||||
import subprocess
|
import subprocess
|
||||||
import sys
|
import sys
|
||||||
import tempfile
|
import tempfile
|
||||||
|
import time
|
||||||
import unittest
|
import unittest
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
@@ -17,10 +20,16 @@ SECURITY = REPO_ROOT / "skills" / "kvb-check-security" / "scripts" / "security.p
|
|||||||
GIT_DIR = os.path.dirname(shutil.which("git"))
|
GIT_DIR = os.path.dirname(shutil.which("git"))
|
||||||
PLACEHOLDER = "placeholder-value-never-shown"
|
PLACEHOLDER = "placeholder-value-never-shown"
|
||||||
FAKE = """#!{shebang}
|
FAKE = """#!{shebang}
|
||||||
import json, os, pathlib, sys
|
import json, os, pathlib, subprocess, sys, time
|
||||||
args = sys.argv[1:]
|
args = sys.argv[1:]
|
||||||
pathlib.Path(__file__).with_name("argv.json").write_text(json.dumps(args))
|
pathlib.Path(__file__).with_name("argv.json").write_text(json.dumps(args))
|
||||||
pathlib.Path(__file__).with_name("home.txt").write_text(os.environ["HOME"])
|
pathlib.Path(__file__).with_name("home.txt").write_text(os.environ["HOME"])
|
||||||
|
pathlib.Path(__file__).with_name("timeout.txt").write_text(os.environ.get("MEDUSA_SCANNER_TIMEOUT", "unset"))
|
||||||
|
if os.environ.get("FAKE_MEDUSA_CHILD"):
|
||||||
|
child = subprocess.Popen([sys.executable, "-c", "import time; time.sleep(60)"])
|
||||||
|
pathlib.Path(__file__).with_name("child.txt").write_text(str(child.pid))
|
||||||
|
pathlib.Path(__file__).with_name("pid.txt").write_text(str(os.getpid()))
|
||||||
|
time.sleep(float(os.environ.get("FAKE_MEDUSA_SLEEP", "0")))
|
||||||
pathlib.Path(os.environ["HOME"], ".medusa").mkdir()
|
pathlib.Path(os.environ["HOME"], ".medusa").mkdir()
|
||||||
target, out = pathlib.Path(args[1]), pathlib.Path(args[args.index("-o") + 1])
|
target, out = pathlib.Path(args[1]), pathlib.Path(args[args.index("-o") + 1])
|
||||||
out.mkdir(parents=True)
|
out.mkdir(parents=True)
|
||||||
@@ -64,6 +73,7 @@ class MedusaConfig:
|
|||||||
exclude_files: List[str] = field(default_factory=lambda: ["*.min.js"])
|
exclude_files: List[str] = field(default_factory=lambda: ["*.min.js"])
|
||||||
"""
|
"""
|
||||||
PARALLEL = """MAX_SCAN_FILE_SIZE = {size}
|
PARALLEL = """MAX_SCAN_FILE_SIZE = {size}
|
||||||
|
PER_SCANNER_TIMEOUT = {timeout}
|
||||||
|
|
||||||
|
|
||||||
class MedusaParallelScanner:
|
class MedusaParallelScanner:
|
||||||
@@ -78,6 +88,29 @@ class MedusaParallelScanner:
|
|||||||
HAND = "review trust boundaries, secrets and injection in the diff by hand, then pass --security '~<n>' to kvb-check-score"
|
HAND = "review trust boundaries, secrets and injection in the diff by hand, then pass --security '~<n>' to kvb-check-score"
|
||||||
|
|
||||||
|
|
||||||
|
def gone(pid: int, wait: float = 5.0) -> bool:
|
||||||
|
until = time.monotonic() + wait
|
||||||
|
while True:
|
||||||
|
try:
|
||||||
|
state = Path(f"/proc/{pid}/stat").read_text().rsplit(")", 1)[1].split()[0]
|
||||||
|
except OSError:
|
||||||
|
return True
|
||||||
|
if state in "ZX":
|
||||||
|
return True
|
||||||
|
if time.monotonic() >= until:
|
||||||
|
return False
|
||||||
|
time.sleep(0.05)
|
||||||
|
|
||||||
|
|
||||||
|
def ended(pids: list[int]) -> None:
|
||||||
|
for pid in pids:
|
||||||
|
if not gone(pid, 0):
|
||||||
|
try:
|
||||||
|
os.kill(pid, signal.SIGKILL)
|
||||||
|
except ProcessLookupError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
class TestSecurity(unittest.TestCase):
|
class TestSecurity(unittest.TestCase):
|
||||||
def setUp(self):
|
def setUp(self):
|
||||||
tmp = tempfile.TemporaryDirectory()
|
tmp = tempfile.TemporaryDirectory()
|
||||||
@@ -117,14 +150,31 @@ class TestSecurity(unittest.TestCase):
|
|||||||
self.git("add", "-A")
|
self.git("add", "-A")
|
||||||
self.git("commit", "-qm", "base")
|
self.git("commit", "-qm", "base")
|
||||||
|
|
||||||
def limits(self, size: str = "2 * 1024 * 1024", lines: str = "50_000") -> None:
|
def limits(
|
||||||
for name, text in (("core/parallel.py", PARALLEL.format(size=size)), ("scanners/base.py", BASE.format(lines=lines))):
|
self, size: str = "2 * 1024 * 1024", lines: str = "50_000",
|
||||||
|
timeout: str = 'int(os.environ.get("MEDUSA_SCANNER_TIMEOUT") or 60)',
|
||||||
|
) -> None:
|
||||||
|
parallel = PARALLEL.format(size=size, timeout=timeout)
|
||||||
|
for name, text in (("core/parallel.py", parallel), ("scanners/base.py", BASE.format(lines=lines))):
|
||||||
(self.package / name).parent.mkdir(parents=True, exist_ok=True)
|
(self.package / name).parent.mkdir(parents=True, exist_ok=True)
|
||||||
(self.package / name).write_text(text, encoding="utf-8")
|
(self.package / name).write_text(text, encoding="utf-8")
|
||||||
|
|
||||||
def git(self, *args: str) -> None:
|
def git(self, *args: str) -> None:
|
||||||
subprocess.run(["git", *args], cwd=self.repo, env=self.env, capture_output=True, check=True)
|
subprocess.run(["git", *args], cwd=self.repo, env=self.env, capture_output=True, check=True)
|
||||||
|
|
||||||
|
def started(self, *names: str) -> list[int]:
|
||||||
|
"""Returns:
|
||||||
|
the pids the fake medusa writes to names (default: its own and its
|
||||||
|
child's), once all are written; all are killed on cleanup.
|
||||||
|
"""
|
||||||
|
markers = [self.bin / name for name in names or ("pid.txt", "child.txt")]
|
||||||
|
until = time.monotonic() + 20
|
||||||
|
while not all(m.exists() and m.read_text() for m in markers) and time.monotonic() < until:
|
||||||
|
time.sleep(0.05)
|
||||||
|
pids = [int(m.read_text()) for m in markers]
|
||||||
|
self.addCleanup(ended, pids)
|
||||||
|
return pids
|
||||||
|
|
||||||
def write(self, path: str, text: str) -> None:
|
def write(self, path: str, text: str) -> None:
|
||||||
(self.repo / path).parent.mkdir(parents=True, exist_ok=True)
|
(self.repo / path).parent.mkdir(parents=True, exist_ok=True)
|
||||||
(self.repo / path).write_text(text, encoding="utf-8")
|
(self.repo / path).write_text(text, encoding="utf-8")
|
||||||
@@ -572,6 +622,134 @@ class TestSecurity(unittest.TestCase):
|
|||||||
)
|
)
|
||||||
(self.repo / name).unlink()
|
(self.repo / name).unlink()
|
||||||
|
|
||||||
|
def test_the_scanner_timeout_reaches_medusa_in_place_of_the_callers_and_takes_whole_seconds_only(self):
|
||||||
|
spec = importlib.util.spec_from_file_location("kvb_check_security", SECURITY)
|
||||||
|
security = importlib.util.module_from_spec(spec)
|
||||||
|
spec.loader.exec_module(security)
|
||||||
|
knob = self.bin / "timeout.txt"
|
||||||
|
cases = (((), str(security.TIMEOUT)), (("--timeout", "86400"), "86400"))
|
||||||
|
for number, (extra, handed) in enumerate(cases, 2):
|
||||||
|
with self.subTest(handed=handed):
|
||||||
|
self.write("app.py", f"print({number})\n")
|
||||||
|
result = self.scan(*extra, env={**self.env, "MEDUSA_SCANNER_TIMEOUT": "5"})
|
||||||
|
self.assertEqual(result.returncode, 0, result.stderr)
|
||||||
|
self.assertEqual(knob.read_text(encoding="utf-8"), handed)
|
||||||
|
for flag in ("--timeout", "--deadline"):
|
||||||
|
for value in ("0", "-1", "1.5", "x", "86401"):
|
||||||
|
with self.subTest(flag=flag, value=value):
|
||||||
|
result = self.scan(flag, value, env=self.env)
|
||||||
|
self.assertEqual(result.returncode, 2)
|
||||||
|
self.assertIn(f"want whole seconds from 1 to 86400, got {value!r}", result.stderr)
|
||||||
|
|
||||||
|
def test_a_scanner_fault_claims_no_cause_medusa_cannot_tell_apart(self):
|
||||||
|
self.write("app.py", "print(2)\n")
|
||||||
|
timed = "1 scanner error(s)/timeout(s) during scan: ModelAttackScanner (1)"
|
||||||
|
unread = "1 director(ies) could not be read and were skipped (e.g. locked)"
|
||||||
|
capped = (
|
||||||
|
"a scanner crashed or hit medusa's fixed 2 s cap (medusa counts both together): rerun when idle; "
|
||||||
|
"if it repeats, give medusa the MEDUSA_SCANNER_TIMEOUT knob"
|
||||||
|
)
|
||||||
|
erred = "scanner error (no scanner reached the 2 s cap): rerun when idle"
|
||||||
|
rerun = "rerun once the host is idle"
|
||||||
|
cases = (
|
||||||
|
("2", "2.2", timed, capped),
|
||||||
|
("2", "0", timed, erred),
|
||||||
|
("2", "2.2", unread, rerun),
|
||||||
|
("config.scanner_timeout()", "2.2", timed, rerun),
|
||||||
|
)
|
||||||
|
for timeout, sleep, line, advice in cases:
|
||||||
|
with self.subTest(timeout=timeout, sleep=sleep, line=line):
|
||||||
|
self.limits(timeout=timeout)
|
||||||
|
result = self.scan(env={**self.env, "FAKE_MEDUSA_STDOUT": f"⚠️ {line}\n", "FAKE_MEDUSA_SLEEP": sleep})
|
||||||
|
problem = f"medusa run incomplete, nothing cached, {advice}: {line}"
|
||||||
|
self.assertEqual((result.stdout, result.returncode), (f"🛡️ security – ({problem}: {HAND})\n", 4))
|
||||||
|
|
||||||
|
def test_a_medusa_run_past_the_deadline_takes_its_whole_group_and_names_the_flag_to_raise(self):
|
||||||
|
self.write("app.py", "print(2)\n")
|
||||||
|
start = time.monotonic()
|
||||||
|
result = self.scan("--deadline", "1", env={**self.env, "FAKE_MEDUSA_SLEEP": "30", "FAKE_MEDUSA_CHILD": "1"})
|
||||||
|
self.assertLess(time.monotonic() - start, 8)
|
||||||
|
problem = (
|
||||||
|
"medusa run incomplete, nothing cached, medusa ran past its 1 s --deadline: "
|
||||||
|
"rerun it in the background with --deadline raised, up to 86400"
|
||||||
|
)
|
||||||
|
self.assertEqual((result.stdout, result.returncode), (f"🛡️ security – ({problem}: {HAND})\n", 4))
|
||||||
|
self.assertTrue(all(gone(pid) for pid in self.started()))
|
||||||
|
self.assertEqual(list((self.base / "council" / "security-cache").glob("*.json")), [])
|
||||||
|
|
||||||
|
def test_a_stop_signal_to_security_py_takes_medusa_and_its_child_along(self):
|
||||||
|
self.write("app.py", "print(2)\n")
|
||||||
|
env = {**self.env, "FAKE_MEDUSA_SLEEP": "30", "FAKE_MEDUSA_CHILD": "1"}
|
||||||
|
for number in (signal.SIGTERM, signal.SIGINT, signal.SIGHUP, signal.SIGKILL):
|
||||||
|
with self.subTest(signal=number.name):
|
||||||
|
for name in ("pid.txt", "child.txt"):
|
||||||
|
(self.bin / name).unlink(missing_ok=True)
|
||||||
|
security = subprocess.Popen(
|
||||||
|
[sys.executable, str(SECURITY), "--repo", str(self.repo), "--base", "HEAD"], cwd=self.base,
|
||||||
|
env=env, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, start_new_session=True,
|
||||||
|
)
|
||||||
|
medusa, child = self.started()
|
||||||
|
os.killpg(security.pid, number)
|
||||||
|
self.assertEqual(security.wait(timeout=20), -number)
|
||||||
|
self.assertTrue(gone(medusa))
|
||||||
|
if number != signal.SIGKILL:
|
||||||
|
self.assertTrue(gone(child))
|
||||||
|
|
||||||
|
def test_the_cache_key_holds_the_timeout_medusa_got_not_the_callers(self):
|
||||||
|
self.write("app.py", "print(2)\n")
|
||||||
|
argv = self.bin / "argv.json"
|
||||||
|
self.scan(env={**self.env, "MEDUSA_SCANNER_TIMEOUT": "5"})
|
||||||
|
argv.unlink()
|
||||||
|
self.scan(env={**self.env, "MEDUSA_SCANNER_TIMEOUT": "6"})
|
||||||
|
self.assertFalse(argv.exists())
|
||||||
|
self.scan("--timeout", "7", env=self.env)
|
||||||
|
self.assertTrue(argv.exists())
|
||||||
|
|
||||||
|
def test_signals_the_caller_ignores_stay_ignored_and_the_scan_scores(self):
|
||||||
|
self.write("app.py", "print(2)\n")
|
||||||
|
ignored = (signal.SIGHUP, signal.SIGINT)
|
||||||
|
previous = {number: signal.signal(number, signal.SIG_IGN) for number in ignored}
|
||||||
|
try:
|
||||||
|
security = subprocess.Popen(
|
||||||
|
[sys.executable, str(SECURITY), "--repo", str(self.repo), "--base", "HEAD"], cwd=self.base,
|
||||||
|
env={**self.env, "FAKE_MEDUSA_SLEEP": "2"}, stdout=subprocess.PIPE, stderr=subprocess.DEVNULL,
|
||||||
|
text=True, start_new_session=True,
|
||||||
|
)
|
||||||
|
finally:
|
||||||
|
for number, handler in previous.items():
|
||||||
|
signal.signal(number, handler)
|
||||||
|
self.started("pid.txt")
|
||||||
|
for number in ignored:
|
||||||
|
os.killpg(security.pid, number)
|
||||||
|
out, _ = security.communicate(timeout=20)
|
||||||
|
self.assertEqual((out, security.returncode), ("🛡️ security 100 % (medusa: 0 findings in 1 file)\n", 0))
|
||||||
|
|
||||||
|
def test_the_first_stop_signal_ignores_the_others_and_kills_the_group(self):
|
||||||
|
spec = importlib.util.spec_from_file_location("kvb_check_security_halt", SECURITY)
|
||||||
|
security = importlib.util.module_from_spec(spec)
|
||||||
|
spec.loader.exec_module(security)
|
||||||
|
for number in security.STOPS:
|
||||||
|
self.addCleanup(signal.signal, number, signal.signal(number, signal.default_int_handler))
|
||||||
|
sleeper = subprocess.Popen([sys.executable, "-c", "import time; time.sleep(60)"], start_new_session=True)
|
||||||
|
self.addCleanup(ended, [sleeper.pid])
|
||||||
|
security.halt([sleeper.pid], security.STOPS)
|
||||||
|
self.assertEqual([signal.getsignal(number) for number in security.STOPS], [signal.SIG_IGN] * len(security.STOPS))
|
||||||
|
self.assertEqual(sleeper.wait(timeout=10), -signal.SIGKILL)
|
||||||
|
|
||||||
|
def test_a_libc_without_prctl_only_loses_the_parent_death_signal(self):
|
||||||
|
self.write("app.py", "print(2)\n")
|
||||||
|
wrap = (
|
||||||
|
"import ctypes, runpy, sys; ctypes.CDLL = lambda *args, **kwargs: object(); "
|
||||||
|
"sys.argv = sys.argv[1:]; runpy.run_path(sys.argv[0], run_name='__main__')"
|
||||||
|
)
|
||||||
|
result = subprocess.run(
|
||||||
|
[sys.executable, "-c", wrap, str(SECURITY), "--repo", str(self.repo), "--base", "HEAD"],
|
||||||
|
cwd=self.base, env=self.env, capture_output=True, text=True, check=False,
|
||||||
|
)
|
||||||
|
self.assertEqual(
|
||||||
|
(result.stdout, result.returncode), ("🛡️ security 100 % (medusa: 0 findings in 1 file)\n", 0), result.stderr,
|
||||||
|
)
|
||||||
|
|
||||||
def test_usage_errors_exit_2(self):
|
def test_usage_errors_exit_2(self):
|
||||||
result = subprocess.run(
|
result = subprocess.run(
|
||||||
[sys.executable, str(SECURITY), "--repo", str(self.repo), "--base", "nope"],
|
[sys.executable, str(SECURITY), "--repo", str(self.repo), "--base", "nope"],
|
||||||
|
|||||||
Reference in new issue
Block a user