docs(requirements): add 009 code graph, 010 security cache, 011 test cache
Token-saving requirements picked by the operator on 2026-11-14: the code graph is initialised automatically per repository, security scan results are cached per file by content, and green test results are reused only under an exact key. Implementation follows once the running lanes land; the shared result store is requirement 003 of the quality repository. 🎯 Fix: 95 % 💥 Break: 1 % 💸 Tokens: 100 % ⏱️ Velocity: 100 % 🛡️ Security: 100 % 🏆 Score: 94 🚦 Gate: ✅ 90/5, tokens and velocity <= 100, security 100 % Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
a2a03995cb
commit
fec77026ea
4 files changed
+538
No files matched your search
@@ -0,0 +1,172 @@
|
||||
# 010 - Security Scan Cache per File
|
||||
|
||||
## User Story
|
||||
|
||||
As an agent running `kvb-check-security` on many branches and sessions, I
|
||||
want the MEDUSA findings of every unchanged file reused under an exact
|
||||
per-file key, so that one changed file rescans only that file and the same
|
||||
file is never scanned twice across branches and sessions.
|
||||
|
||||
## Decisions (operator-confirmed 2026-11-14; do not re-litigate)
|
||||
|
||||
1. **Depends on**: quality 003 (the result store). Independent of 009 and
|
||||
011.
|
||||
2. **Timing**: implementation starts once the running agent lanes are done
|
||||
and quality 003 has shipped its store.
|
||||
3. **Today, measured**: `keyed()` in `scripts/security.py` hashes the script,
|
||||
the medusa binary's path, mtime and size, `PATH`, the files not covered,
|
||||
every `MEDUSA_*` variable and, per copied file, its tree path, repository
|
||||
path, mode and content; one entry per whole diff lives in
|
||||
`<council root>/security-cache/` for `CACHE_SECONDS` (24 h), and medusa
|
||||
runs with `--no-cache`. One changed file therefore rescans the whole diff,
|
||||
and an unchanged file on another branch or in another session is scanned
|
||||
again; the ModelAttackScanner needs 71.8 s for one 2949-line file.
|
||||
4. **Key per file**: the sha256 of the file content, the medusa version
|
||||
(2026.7.0 today, from its dist-info), a digest of medusa's rule set (its
|
||||
`rules/` tree and the RULES that `security.py` reads from medusa's
|
||||
source) and the digest of `security.py`. No time expiry: the key changes
|
||||
whenever anything relevant changes. Rejected: the whole-diff key (today's
|
||||
behaviour); a 24 h expiry (a key that changes with every input needs
|
||||
none).
|
||||
5. **What selects and runs a scanner joins the key (proposed)**: medusa
|
||||
picks scanners by extension, exact name, prefix and parent folder name
|
||||
(`ai_context_scanner`, `rag_security_scanner`, `mcp_config_scanner` and
|
||||
`agent_memory_scanner` read `file_path.parent.name`), and most scanners
|
||||
call external tools found on `PATH` (shellcheck, semgrep, gitleaks, trivy
|
||||
and more). The key therefore also holds the scanned tree path, the file
|
||||
mode, every `MEDUSA_*` variable and the resolved path, size and mtime of
|
||||
each external tool, as today's key holds `PATH`. Without them a moved file
|
||||
or a newly installed tool would reuse findings it never produced.
|
||||
6. **Cross-file check first**: before any per-file entry is written, prove
|
||||
that medusa's findings for a file do not depend on other files of the
|
||||
scan tree; where a cross-file rule exists, the files it reads stay per
|
||||
diff. Found while drafting, medusa 2026.7.0: `trivy_scanner` runs `trivy
|
||||
fs` on the file's parent folder, `rust_scanner` walks up to `Cargo.toml`,
|
||||
`_find_config_file` in `scanners/base.py` walks up for tool
|
||||
configuration past the scratch tree into `/tmp`, and
|
||||
`docker_compose_scanner` runs in the file's folder. `medusa scan` has no
|
||||
flag that selects scanners, so a file that `FILE_SCANNERS` routes to a
|
||||
cross-file scanner stays in a per-diff group, keyed like today minus the
|
||||
expiry. The check is an acceptance criterion.
|
||||
7. **Through the quality store**: entries are written and read with
|
||||
`quality store put` and `get` (quality 003) under tool `medusa`;
|
||||
`<council root>/security-cache/` goes away. A payload holds per finding
|
||||
only `rule`, `line` and `severity`, as quality decision 23 allows. `sha1`
|
||||
and `context` are line hashes, which decision 23 keeps out of stored
|
||||
data, so `security.py` recomputes them from the file after a hit; the file
|
||||
content is part of the key, so they come out identical.
|
||||
8. **Incomplete scans are never cached**: a run with a non-zero exit, a FAULT
|
||||
or SKIP line, a `files_scanned` or `lines_scanned` shortfall, an
|
||||
unreadable report or a file never covered writes no entry for any of its
|
||||
files, as `scan()` does today.
|
||||
9. **Accepted findings unchanged**: `security-accepted.jsonl`, `accept`, the
|
||||
rule, path, sha1 and context match and the score formula stay as they
|
||||
are.
|
||||
10. **A store failure costs time only**: a missing quality binary or a store
|
||||
miss on error runs medusa as today; it never yields a score without a
|
||||
scan.
|
||||
11. **Rejected: radio digest**. The operator rejected it on 2026-11-14:
|
||||
every radio message stays in full in the hooks, because each one is sent
|
||||
for a reason.
|
||||
|
||||
## Approach
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
subgraph P["🧭 0 · Preparation"]
|
||||
P1["📋 Write requirement"] --> P2{"✅ Operator approves?"}
|
||||
P2 -- "✏️ corrections" --> P1
|
||||
end
|
||||
P2 -- "🤖 yes" --> A1
|
||||
subgraph A["🔬 A · Cross-file check"]
|
||||
A1["🧪 Scan fixtures alone and together"]
|
||||
A1 --> A2["📋 List cross-file routes"]
|
||||
A2 --> AF{"💬 Operator agrees with the list?"}
|
||||
AF -- "✏️ changes" --> A1
|
||||
end
|
||||
AF -- "👍 fits" --> B0
|
||||
subgraph B["🗝️ B · Per-file key and store"]
|
||||
B0{"📦 quality 003 store shipped?"}
|
||||
B0 -- "⏳ no: wait" --> B0
|
||||
B0 -- "✅ yes" --> B1["🧩 Key, payload, store calls"]
|
||||
B1 --> B2{"🧪 Unit and integration green?"}
|
||||
B2 -- "🔴 no: fix" --> B1
|
||||
B2 -- "🟢 yes" --> B3{"⏱️ Unchanged diff under 2 s?"}
|
||||
B3 -- "🔴 no: fix" --> B1
|
||||
B3 -- "🟢 yes" --> BF{"💬 Operator feedback?"}
|
||||
BF -- "✏️ changes" --> B1
|
||||
end
|
||||
BF -- "👍 fits" --> D1["🏁 Review, commit with operator OK, hand-over"]
|
||||
```
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
### Cross-file check
|
||||
|
||||
- [ ] `test_findings_file_local`: for a fixture set covering every
|
||||
`FILE_SCANNERS` route, each file's findings scanned alone equal its
|
||||
findings scanned inside the full fixture tree; every route where they
|
||||
differ is listed in `CROSS_FILE` in `security.py`.
|
||||
- [ ] `CROSS_FILE` holds the routes of `trivy_scanner`, `rust_scanner` and
|
||||
`docker_compose_scanner` and the configuration names that
|
||||
`_find_config_file` looks up, unless `test_findings_file_local` proves
|
||||
one of them file-local.
|
||||
- [ ] `test_cross_file_group_rescans`: changing a sibling of a file in the
|
||||
per-diff group (a `Cargo.toml` next to a `.rs` file) rescans the group.
|
||||
|
||||
### Key
|
||||
|
||||
- [ ] `test_key_per_file`: with 10 scanned files of which 1 changed, medusa's
|
||||
scratch tree holds that 1 file and the other 9 come from the store.
|
||||
- [ ] `test_key_across_branches`: an unchanged file on a second branch,
|
||||
scanned by a second process, hits without running medusa.
|
||||
- [ ] `test_key_parts`: each of content, medusa version, rule set digest,
|
||||
`security.py` digest, tree path, mode, a `MEDUSA_*` variable and an
|
||||
external tool's mtime changes the key.
|
||||
- [ ] `rg -n CACHE_SECONDS scripts/security.py` finds nothing.
|
||||
- [ ] The medusa command line still carries `--no-cache`.
|
||||
|
||||
### Store
|
||||
|
||||
- [ ] `test_store_payload_fields`: every stored finding holds exactly `rule`,
|
||||
`line` and `severity`.
|
||||
- [ ] `test_hit_fingerprints_match`: `sha1` and `context` of a hit equal those
|
||||
of a fresh scan of the same file.
|
||||
- [ ] `test_no_legacy_cache_dir`: a scan writes nothing under
|
||||
`<council root>/security-cache/`.
|
||||
- [ ] `test_store_missing_runs_scan`: without the quality binary, or with a
|
||||
store that errors, the scan runs and its output equals today's.
|
||||
|
||||
### Incomplete scans and accepted findings
|
||||
|
||||
- [ ] `test_incomplete_caches_nothing`: a FAULT line, a non-zero exit, a
|
||||
`lines_scanned` shortfall, a SKIP line and an unreadable report each
|
||||
leave zero store entries for every file of that run.
|
||||
- [ ] The existing accepted-findings tests in `tests/test_security.py` pass
|
||||
unchanged, and `test_hit_accepts_like_scan`: a hit with an accepted
|
||||
finding leaves the score as a fresh scan does.
|
||||
- [ ] `test_hit_output_identical`: output line, `--json` and exit code of a
|
||||
hit equal those of a fresh scan of the same diff.
|
||||
|
||||
### Speed and scope
|
||||
|
||||
- [ ] A second `security.py` run over an unchanged diff holding the
|
||||
2949-line file that took 71.8 s finishes in under 2 s (`time`).
|
||||
- [ ] The implementation's diff touches no file of `kvb-org-radio`.
|
||||
|
||||
### Tests
|
||||
|
||||
- **Unit**
|
||||
- [ ] `test_key_parts`, `test_store_payload_fields` and the `CROSS_FILE`
|
||||
routing of `taken()` results.
|
||||
- **Integration**
|
||||
- [ ] Real medusa on fixture trees with a temporary `XDG_CACHE_HOME`:
|
||||
`test_findings_file_local`, `test_key_per_file`,
|
||||
`test_key_across_branches`, `test_cross_file_group_rescans`,
|
||||
`test_incomplete_caches_nothing`, `test_hit_fingerprints_match`.
|
||||
- **E2E**
|
||||
- [ ] `tests/e2e/test_security_cache_e2e.py`: two worktrees of skills on
|
||||
branches sharing unchanged files; after the second `security.py` run,
|
||||
`quality store stats --tool medusa` counts new rows only for the
|
||||
files the second branch changed, and its score equals a run with the
|
||||
store removed.
|
||||
Reference in new issue
Block a user