docs(requirements): add 009 code graph, 010 security cache, 011 test cache

Token-saving requirements picked by the operator on 2026-11-14: the code
graph is initialised automatically per repository, security scan results
are cached per file by content, and green test results are reused only
under an exact key. Implementation follows once the running lanes land;
the shared result store is requirement 003 of the quality repository.

🎯 Fix: 95 %
💥 Break: 1 %
💸 Tokens: 100 %
⏱️ Velocity: 100 %
🛡️ Security: 100 %
🏆 Score: 94
🚦 Gate: ✅ 90/5, tokens and velocity <= 100, security 100 %

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Kevin Veen-BirkenbachandClaude Opus 5.5 committed 2026-11-14 14:10:53 +00:00
1 parent a2a03995cb
commit fec77026ea
4 files changed
+538

No files matched your search

@@ -0,0 +1,172 @@
# 010 - Security Scan Cache per File
## User Story
As an agent running `kvb-check-security` on many branches and sessions, I
want the MEDUSA findings of every unchanged file reused under an exact
per-file key, so that one changed file rescans only that file and the same
file is never scanned twice across branches and sessions.
## Decisions (operator-confirmed 2026-11-14; do not re-litigate)
1. **Depends on**: quality 003 (the result store). Independent of 009 and
011.
2. **Timing**: implementation starts once the running agent lanes are done
and quality 003 has shipped its store.
3. **Today, measured**: `keyed()` in `scripts/security.py` hashes the script,
the medusa binary's path, mtime and size, `PATH`, the files not covered,
every `MEDUSA_*` variable and, per copied file, its tree path, repository
path, mode and content; one entry per whole diff lives in
`<council root>/security-cache/` for `CACHE_SECONDS` (24 h), and medusa
runs with `--no-cache`. One changed file therefore rescans the whole diff,
and an unchanged file on another branch or in another session is scanned
again; the ModelAttackScanner needs 71.8 s for one 2949-line file.
4. **Key per file**: the sha256 of the file content, the medusa version
(2026.7.0 today, from its dist-info), a digest of medusa's rule set (its
`rules/` tree and the RULES that `security.py` reads from medusa's
source) and the digest of `security.py`. No time expiry: the key changes
whenever anything relevant changes. Rejected: the whole-diff key (today's
behaviour); a 24 h expiry (a key that changes with every input needs
none).
5. **What selects and runs a scanner joins the key (proposed)**: medusa
picks scanners by extension, exact name, prefix and parent folder name
(`ai_context_scanner`, `rag_security_scanner`, `mcp_config_scanner` and
`agent_memory_scanner` read `file_path.parent.name`), and most scanners
call external tools found on `PATH` (shellcheck, semgrep, gitleaks, trivy
and more). The key therefore also holds the scanned tree path, the file
mode, every `MEDUSA_*` variable and the resolved path, size and mtime of
each external tool, as today's key holds `PATH`. Without them a moved file
or a newly installed tool would reuse findings it never produced.
6. **Cross-file check first**: before any per-file entry is written, prove
that medusa's findings for a file do not depend on other files of the
scan tree; where a cross-file rule exists, the files it reads stay per
diff. Found while drafting, medusa 2026.7.0: `trivy_scanner` runs `trivy
fs` on the file's parent folder, `rust_scanner` walks up to `Cargo.toml`,
`_find_config_file` in `scanners/base.py` walks up for tool
configuration past the scratch tree into `/tmp`, and
`docker_compose_scanner` runs in the file's folder. `medusa scan` has no
flag that selects scanners, so a file that `FILE_SCANNERS` routes to a
cross-file scanner stays in a per-diff group, keyed like today minus the
expiry. The check is an acceptance criterion.
7. **Through the quality store**: entries are written and read with
`quality store put` and `get` (quality 003) under tool `medusa`;
`<council root>/security-cache/` goes away. A payload holds per finding
only `rule`, `line` and `severity`, as quality decision 23 allows. `sha1`
and `context` are line hashes, which decision 23 keeps out of stored
data, so `security.py` recomputes them from the file after a hit; the file
content is part of the key, so they come out identical.
8. **Incomplete scans are never cached**: a run with a non-zero exit, a FAULT
or SKIP line, a `files_scanned` or `lines_scanned` shortfall, an
unreadable report or a file never covered writes no entry for any of its
files, as `scan()` does today.
9. **Accepted findings unchanged**: `security-accepted.jsonl`, `accept`, the
rule, path, sha1 and context match and the score formula stay as they
are.
10. **A store failure costs time only**: a missing quality binary or a store
miss on error runs medusa as today; it never yields a score without a
scan.
11. **Rejected: radio digest**. The operator rejected it on 2026-11-14:
every radio message stays in full in the hooks, because each one is sent
for a reason.
## Approach
```mermaid
flowchart TD
subgraph P["🧭 0 · Preparation"]
P1["📋 Write requirement"] --> P2{"✅ Operator approves?"}
P2 -- "✏️ corrections" --> P1
end
P2 -- "🤖 yes" --> A1
subgraph A["🔬 A · Cross-file check"]
A1["🧪 Scan fixtures alone and together"]
A1 --> A2["📋 List cross-file routes"]
A2 --> AF{"💬 Operator agrees with the list?"}
AF -- "✏️ changes" --> A1
end
AF -- "👍 fits" --> B0
subgraph B["🗝️ B · Per-file key and store"]
B0{"📦 quality 003 store shipped?"}
B0 -- "⏳ no: wait" --> B0
B0 -- "✅ yes" --> B1["🧩 Key, payload, store calls"]
B1 --> B2{"🧪 Unit and integration green?"}
B2 -- "🔴 no: fix" --> B1
B2 -- "🟢 yes" --> B3{"⏱️ Unchanged diff under 2 s?"}
B3 -- "🔴 no: fix" --> B1
B3 -- "🟢 yes" --> BF{"💬 Operator feedback?"}
BF -- "✏️ changes" --> B1
end
BF -- "👍 fits" --> D1["🏁 Review, commit with operator OK, hand-over"]
```
## Acceptance Criteria
### Cross-file check
- [ ] `test_findings_file_local`: for a fixture set covering every
`FILE_SCANNERS` route, each file's findings scanned alone equal its
findings scanned inside the full fixture tree; every route where they
differ is listed in `CROSS_FILE` in `security.py`.
- [ ] `CROSS_FILE` holds the routes of `trivy_scanner`, `rust_scanner` and
`docker_compose_scanner` and the configuration names that
`_find_config_file` looks up, unless `test_findings_file_local` proves
one of them file-local.
- [ ] `test_cross_file_group_rescans`: changing a sibling of a file in the
per-diff group (a `Cargo.toml` next to a `.rs` file) rescans the group.
### Key
- [ ] `test_key_per_file`: with 10 scanned files of which 1 changed, medusa's
scratch tree holds that 1 file and the other 9 come from the store.
- [ ] `test_key_across_branches`: an unchanged file on a second branch,
scanned by a second process, hits without running medusa.
- [ ] `test_key_parts`: each of content, medusa version, rule set digest,
`security.py` digest, tree path, mode, a `MEDUSA_*` variable and an
external tool's mtime changes the key.
- [ ] `rg -n CACHE_SECONDS scripts/security.py` finds nothing.
- [ ] The medusa command line still carries `--no-cache`.
### Store
- [ ] `test_store_payload_fields`: every stored finding holds exactly `rule`,
`line` and `severity`.
- [ ] `test_hit_fingerprints_match`: `sha1` and `context` of a hit equal those
of a fresh scan of the same file.
- [ ] `test_no_legacy_cache_dir`: a scan writes nothing under
`<council root>/security-cache/`.
- [ ] `test_store_missing_runs_scan`: without the quality binary, or with a
store that errors, the scan runs and its output equals today's.
### Incomplete scans and accepted findings
- [ ] `test_incomplete_caches_nothing`: a FAULT line, a non-zero exit, a
`lines_scanned` shortfall, a SKIP line and an unreadable report each
leave zero store entries for every file of that run.
- [ ] The existing accepted-findings tests in `tests/test_security.py` pass
unchanged, and `test_hit_accepts_like_scan`: a hit with an accepted
finding leaves the score as a fresh scan does.
- [ ] `test_hit_output_identical`: output line, `--json` and exit code of a
hit equal those of a fresh scan of the same diff.
### Speed and scope
- [ ] A second `security.py` run over an unchanged diff holding the
2949-line file that took 71.8 s finishes in under 2 s (`time`).
- [ ] The implementation's diff touches no file of `kvb-org-radio`.
### Tests
- **Unit**
- [ ] `test_key_parts`, `test_store_payload_fields` and the `CROSS_FILE`
routing of `taken()` results.
- **Integration**
- [ ] Real medusa on fixture trees with a temporary `XDG_CACHE_HOME`:
`test_findings_file_local`, `test_key_per_file`,
`test_key_across_branches`, `test_cross_file_group_rescans`,
`test_incomplete_caches_nothing`, `test_hit_fingerprints_match`.
- **E2E**
- [ ] `tests/e2e/test_security_cache_e2e.py`: two worktrees of skills on
branches sharing unchanged files; after the second `security.py` run,
`quality store stats --tool medusa` counts new rows only for the
files the second branch changed, and its score equals a run with the
store removed.