perf(skills): load the advocate checks and break-risk method only when needed
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
6544f01eef
commit
20a47e3916
5 files changed
+152
-125
No files matched your search
@@ -45,39 +45,9 @@ never a blocker of the commit at hand.
|
|||||||
|
|
||||||
## The checks
|
## The checks
|
||||||
|
|
||||||
1. **Inbound code** - every file or block of foreign origin carries a licence
|
Only for a target in scope: read `checks.md` next to this file and apply its
|
||||||
that the project licence can absorb in the direction of distribution. GPL
|
seven checks (inbound code, dependency licences, project licence fit,
|
||||||
code cannot enter a project distributed under MIT without relicensing the
|
attribution, copied snippets, data protection, trademarks).
|
||||||
whole; Apache-2.0 cannot enter GPL-2.0-only; code without a licence is all
|
|
||||||
rights reserved.
|
|
||||||
2. **Dependency licences** - for each added or bumped dependency, read the
|
|
||||||
licence of the resolved version (lockfile, registry metadata, or the
|
|
||||||
package's own LICENSE), including the transitive dependencies the lockfile
|
|
||||||
newly pulls. Watch copyleft reach (GPL, AGPL's network clause, LGPL static
|
|
||||||
versus dynamic linking), source-available terms that are not open source
|
|
||||||
(SSPL, BUSL, Commons Clause, "non-commercial"), and an upstream relicensing
|
|
||||||
between the old and the new version.
|
|
||||||
3. **Project licence fit** - the project's declarations still hold against
|
|
||||||
what checks 1 and 2 found: a dependency that forces copyleft forces either
|
|
||||||
the project licence or the dependency's removal.
|
|
||||||
4. **Attribution** - retained copyright notices, the licence text shipped
|
|
||||||
where required, an Apache-2.0 NOTICE propagated, modified-file notices
|
|
||||||
(Apache-2.0 §4(b), GPL-3.0 §5(a)), SPDX headers where the project uses them,
|
|
||||||
credit for CC BY material.
|
|
||||||
5. **Copied snippets** - Stack Overflow content is CC BY-SA (attribution plus
|
|
||||||
share-alike); gists, blog posts, other repositories, and verbatim
|
|
||||||
reproductions of well-known code need a traceable origin and licence.
|
|
||||||
Trivial code without originality (one-liners, usage dictated by an API) is
|
|
||||||
not a finding.
|
|
||||||
6. **Data protection (GDPR, TDDDG)** - new collection, storage, logging, or
|
|
||||||
transfer of personal data; IP addresses sent to third parties through
|
|
||||||
external fonts, CDNs, analytics, or embeds; storage on the user's device
|
|
||||||
without consent (§ 25 TDDDG); transfers outside the EU. Name the data flow
|
|
||||||
and the missing legal basis or consent mechanism.
|
|
||||||
7. **Trademarks and branding** - third-party logos, icons, and product names
|
|
||||||
used beyond plain reference: implied endorsement, use in the project's own
|
|
||||||
name, modified logos. Icon sets carry their own licence, and a free licence
|
|
||||||
on an icon does not license the trademark it depicts.
|
|
||||||
|
|
||||||
## Severity
|
## Severity
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,37 @@
|
|||||||
|
# Advocate: the checks
|
||||||
|
|
||||||
|
Apply each check to a target the `advocate` skill found in scope.
|
||||||
|
|
||||||
|
1. **Inbound code** - every file or block of foreign origin carries a licence
|
||||||
|
that the project licence can absorb in the direction of distribution. GPL
|
||||||
|
code cannot enter a project distributed under MIT without relicensing the
|
||||||
|
whole; Apache-2.0 cannot enter GPL-2.0-only; code without a licence is all
|
||||||
|
rights reserved.
|
||||||
|
2. **Dependency licences** - for each added or bumped dependency, read the
|
||||||
|
licence of the resolved version (lockfile, registry metadata, or the
|
||||||
|
package's own LICENSE), including the transitive dependencies the lockfile
|
||||||
|
newly pulls. Watch copyleft reach (GPL, AGPL's network clause, LGPL static
|
||||||
|
versus dynamic linking), source-available terms that are not open source
|
||||||
|
(SSPL, BUSL, Commons Clause, "non-commercial"), and an upstream relicensing
|
||||||
|
between the old and the new version.
|
||||||
|
3. **Project licence fit** - the project's declarations still hold against
|
||||||
|
what checks 1 and 2 found: a dependency that forces copyleft forces either
|
||||||
|
the project licence or the dependency's removal.
|
||||||
|
4. **Attribution** - retained copyright notices, the licence text shipped
|
||||||
|
where required, an Apache-2.0 NOTICE propagated, modified-file notices
|
||||||
|
(Apache-2.0 §4(b), GPL-3.0 §5(a)), SPDX headers where the project uses them,
|
||||||
|
credit for CC BY material.
|
||||||
|
5. **Copied snippets** - Stack Overflow content is CC BY-SA (attribution plus
|
||||||
|
share-alike); gists, blog posts, other repositories, and verbatim
|
||||||
|
reproductions of well-known code need a traceable origin and licence.
|
||||||
|
Trivial code without originality (one-liners, usage dictated by an API) is
|
||||||
|
not a finding.
|
||||||
|
6. **Data protection (GDPR, TDDDG)** - new collection, storage, logging, or
|
||||||
|
transfer of personal data; IP addresses sent to third parties through
|
||||||
|
external fonts, CDNs, analytics, or embeds; storage on the user's device
|
||||||
|
without consent (§ 25 TDDDG); transfers outside the EU. Name the data flow
|
||||||
|
and the missing legal basis or consent mechanism.
|
||||||
|
7. **Trademarks and branding** - third-party logos, icons, and product names
|
||||||
|
used beyond plain reference: implied endorsement, use in the project's own
|
||||||
|
name, modified logos. Icon sets carry their own licence, and a free licence
|
||||||
|
on an icon does not license the trademark it depicts.
|
||||||
+24
-75
@@ -9,86 +9,35 @@ description: >
|
|||||||
|
|
||||||
Break risk is the probability the change breaks something it did not intend to
|
Break risk is the probability the change breaks something it did not intend to
|
||||||
touch. It is not the inverse of fix confidence, and it is not a mood. Derive it
|
touch. It is not the inverse of fix confidence, and it is not a mood. Derive it
|
||||||
by answering the questions below; a number produced without them is a guess
|
by the steps below; a number produced without them is a guess wearing a
|
||||||
wearing a percent sign.
|
percent sign.
|
||||||
|
|
||||||
## The consumer question, first and always
|
## Fast path
|
||||||
|
|
||||||
Enumerate every consumer of every thing the change alters, before naming a
|
A change whose files nothing builds, deploys, or runs (documentation and
|
||||||
number. A written caveat never licenses a low number; an enumeration does.
|
comments only, or tests that ran green with it) has no consumers outside
|
||||||
|
itself. Name the files (`git diff --cached --name-only`), report break risk
|
||||||
|
under 1%, and skip the method. Any other file takes the full method.
|
||||||
|
|
||||||
- Search the **committed** state, not just the working tree. A symbol that looks
|
## The method
|
||||||
unused in your branch may still be called from a file you have not opened
|
|
||||||
(`git grep <symbol> HEAD`). Removing a function with zero external callers is
|
|
||||||
a different risk class from removing one with nine.
|
|
||||||
- Enumerate consumers of the **output**, not only of the code: who reads the
|
|
||||||
variable, renders the template, parses the emitted block, imports the
|
|
||||||
constant.
|
|
||||||
- Say the count out loud. "Two consumers, both inside this change set" is
|
|
||||||
evidence. "I don't think anything else uses it" is not.
|
|
||||||
|
|
||||||
## Measure the delta at the surface that matters
|
1. **Consumers**: enumerate every consumer of everything the change alters, in
|
||||||
|
the committed state (`git grep <symbol> HEAD`) and of its output; say the
|
||||||
|
count. A caveat never licenses a low number; an enumeration does.
|
||||||
|
2. **Delta**: run the old and the new code on today's inputs and compare at
|
||||||
|
the surface where the difference shows (rendered output, emitted config,
|
||||||
|
returned set, wire bytes), not as the source reads.
|
||||||
|
3. **Live or latent**: per finding, whether a shipped configuration reaches it.
|
||||||
|
Only live findings move the number materially.
|
||||||
|
4. **Matrix**: a verdict per deployment axis the project has, never averaged.
|
||||||
|
5. **Failure mode**: fails closed and loud, or open and silent; the latter can
|
||||||
|
outrank a likelier failure.
|
||||||
|
6. **Evidence**: executed beats derived; a report, a summary or your own
|
||||||
|
earlier message is not evidence; negative-control every check you cite,
|
||||||
|
including one you did not write.
|
||||||
|
|
||||||
Compare what the old code **actually produced** against what the new code
|
`reference.md` next to this file explains each step with the trap it defeats.
|
||||||
produces, at the point where the difference becomes observable: rendered output,
|
Read it when a step is new to you or its result is contested.
|
||||||
emitted config, returned set, wire bytes.
|
|
||||||
|
|
||||||
The trap this defeats: reasoning about what the old code *looks like* it did. A
|
|
||||||
sweep that appeared to forward sixteen ports forwarded **zero** in production,
|
|
||||||
because the flag it gated on was unrendered template text. Had the delta been
|
|
||||||
argued from the source rather than measured, the risk would have been sized
|
|
||||||
against a behaviour that never existed.
|
|
||||||
|
|
||||||
Extract the old implementation from version control and run it on today's
|
|
||||||
inputs. If `DROPPED` is empty, nothing can regress; say so and move on.
|
|
||||||
|
|
||||||
## Classify every finding as live or latent
|
|
||||||
|
|
||||||
For each way the change could break something, decide which it is and say so:
|
|
||||||
|
|
||||||
- **Live**: a shipped configuration, inventory, variant or default reaches it.
|
|
||||||
- **Latent**: reachable only under a configuration nobody ships today.
|
|
||||||
|
|
||||||
Both belong in the report. Only live findings drive the number up materially. A
|
|
||||||
latent finding with a named trigger is worth more to the operator than a vague
|
|
||||||
worry, and it is honest about why the number is not zero.
|
|
||||||
|
|
||||||
## Walk the deployment matrix
|
|
||||||
|
|
||||||
A change verified in one mode is unverified in the others. Enumerate the axes
|
|
||||||
the project actually has, and state the verdict per cell rather than averaging
|
|
||||||
them away. Typical axes: deployment mode, node role, topology, distro,
|
|
||||||
filesystem, single versus multi host.
|
|
||||||
|
|
||||||
Two numbers for two modes beats one blended number that is wrong in both.
|
|
||||||
|
|
||||||
## Weigh the failure mode, not only the probability
|
|
||||||
|
|
||||||
A low-probability failure that **fails open** can outrank a likely one that
|
|
||||||
fails closed. Ask what the world looks like when the change misbehaves:
|
|
||||||
|
|
||||||
- Fails closed and loud: the service refuses to start, a gate goes red, a
|
|
||||||
request errors. Cheap to detect, bounded blast radius.
|
|
||||||
- Fails open and silent: a port stays reachable, a check passes vacuously, a
|
|
||||||
credential keeps working, traffic is forwarded to an unintended place.
|
|
||||||
|
|
||||||
Say which one applies. When a design choice is between an unverifiable control
|
|
||||||
over an exposure and not creating the exposure, prefer the second and say why.
|
|
||||||
|
|
||||||
## Evidence rules
|
|
||||||
|
|
||||||
- Executed beats derived. Run it, render it, read the output.
|
|
||||||
- A subagent's report, a conversation summary, or your own earlier message is
|
|
||||||
**not** evidence. Re-verify the claim yourself before it changes a number.
|
|
||||||
- Negative control before claiming a check works: make the thing it should
|
|
||||||
catch, watch it fail, restore. A check never observed failing is not known to
|
|
||||||
work.
|
|
||||||
- The same applies to a check you did not write. Citing an existing lint, test
|
|
||||||
or gate as the reason a change is safe means reading its pattern, not its
|
|
||||||
docstring, and watching it fail on the case you are trusting it to catch. A
|
|
||||||
green gate whose rule never matched your case is silence, not a pass.
|
|
||||||
- After a context compaction, rebuild attributions from the source; conclusions
|
|
||||||
survive compaction, the evidence under them does not.
|
|
||||||
|
|
||||||
## Producing the number
|
## Producing the number
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,81 @@
|
|||||||
|
# Break risk: the method in detail
|
||||||
|
|
||||||
|
Each step of the `break-risk` skill, with the trap it defeats.
|
||||||
|
|
||||||
|
## The consumer question, first and always
|
||||||
|
|
||||||
|
Enumerate every consumer of every thing the change alters, before naming a
|
||||||
|
number. A written caveat never licenses a low number; an enumeration does.
|
||||||
|
|
||||||
|
- Search the **committed** state, not just the working tree. A symbol that looks
|
||||||
|
unused in your branch may still be called from a file you have not opened
|
||||||
|
(`git grep <symbol> HEAD`). Removing a function with zero external callers is
|
||||||
|
a different risk class from removing one with nine.
|
||||||
|
- Enumerate consumers of the **output**, not only of the code: who reads the
|
||||||
|
variable, renders the template, parses the emitted block, imports the
|
||||||
|
constant.
|
||||||
|
- Say the count out loud. "Two consumers, both inside this change set" is
|
||||||
|
evidence. "I don't think anything else uses it" is not.
|
||||||
|
|
||||||
|
## Measure the delta at the surface that matters
|
||||||
|
|
||||||
|
Compare what the old code **actually produced** against what the new code
|
||||||
|
produces, at the point where the difference becomes observable: rendered output,
|
||||||
|
emitted config, returned set, wire bytes.
|
||||||
|
|
||||||
|
The trap this defeats: reasoning about what the old code *looks like* it did. A
|
||||||
|
sweep that appeared to forward sixteen ports forwarded **zero** in production,
|
||||||
|
because the flag it gated on was unrendered template text. Had the delta been
|
||||||
|
argued from the source rather than measured, the risk would have been sized
|
||||||
|
against a behaviour that never existed.
|
||||||
|
|
||||||
|
Extract the old implementation from version control and run it on today's
|
||||||
|
inputs. If `DROPPED` is empty, nothing can regress; say so and move on.
|
||||||
|
|
||||||
|
## Classify every finding as live or latent
|
||||||
|
|
||||||
|
For each way the change could break something, decide which it is and say so:
|
||||||
|
|
||||||
|
- **Live**: a shipped configuration, inventory, variant or default reaches it.
|
||||||
|
- **Latent**: reachable only under a configuration nobody ships today.
|
||||||
|
|
||||||
|
Both belong in the report. Only live findings drive the number up materially. A
|
||||||
|
latent finding with a named trigger is worth more to the operator than a vague
|
||||||
|
worry, and it is honest about why the number is not zero.
|
||||||
|
|
||||||
|
## Walk the deployment matrix
|
||||||
|
|
||||||
|
A change verified in one mode is unverified in the others. Enumerate the axes
|
||||||
|
the project actually has, and state the verdict per cell rather than averaging
|
||||||
|
them away. Typical axes: deployment mode, node role, topology, distro,
|
||||||
|
filesystem, single versus multi host.
|
||||||
|
|
||||||
|
Two numbers for two modes beats one blended number that is wrong in both.
|
||||||
|
|
||||||
|
## Weigh the failure mode, not only the probability
|
||||||
|
|
||||||
|
A low-probability failure that **fails open** can outrank a likely one that
|
||||||
|
fails closed. Ask what the world looks like when the change misbehaves:
|
||||||
|
|
||||||
|
- Fails closed and loud: the service refuses to start, a gate goes red, a
|
||||||
|
request errors. Cheap to detect, bounded blast radius.
|
||||||
|
- Fails open and silent: a port stays reachable, a check passes vacuously, a
|
||||||
|
credential keeps working, traffic is forwarded to an unintended place.
|
||||||
|
|
||||||
|
Say which one applies. When a design choice is between an unverifiable control
|
||||||
|
over an exposure and not creating the exposure, prefer the second and say why.
|
||||||
|
|
||||||
|
## Evidence rules
|
||||||
|
|
||||||
|
- Executed beats derived. Run it, render it, read the output.
|
||||||
|
- A subagent's report, a conversation summary, or your own earlier message is
|
||||||
|
**not** evidence. Re-verify the claim yourself before it changes a number.
|
||||||
|
- Negative control before claiming a check works: make the thing it should
|
||||||
|
catch, watch it fail, restore. A check never observed failing is not known to
|
||||||
|
work.
|
||||||
|
- The same applies to a check you did not write. Citing an existing lint, test
|
||||||
|
or gate as the reason a change is safe means reading its pattern, not its
|
||||||
|
docstring, and watching it fail on the case you are trusting it to catch. A
|
||||||
|
green gate whose rule never matched your case is silence, not a pass.
|
||||||
|
- After a context compaction, rebuild attributions from the source; conclusions
|
||||||
|
survive compaction, the evidence under them does not.
|
||||||
@@ -10,11 +10,8 @@ description: >
|
|||||||
Every stage summary, fix report, or ready-to-commit bundle ends with a
|
Every stage summary, fix report, or ready-to-commit bundle ends with a
|
||||||
confidence block.
|
confidence block.
|
||||||
|
|
||||||
Invoke the `break-risk` skill every time, before writing the block. The break
|
Invoke the `break-risk` skill every time, before writing the block; this skill
|
||||||
risk half is derived there — consumers enumerated, the delta measured at the
|
reports the number it derives and never estimates one itself.
|
||||||
surface where it shows, live separated from latent, the deployment matrix
|
|
||||||
walked, the failure mode weighed. Do not estimate it here; this skill reports
|
|
||||||
the number that skill produces.
|
|
||||||
|
|
||||||
1. **Fix confidence** (percent): probability the change does what it
|
1. **Fix confidence** (percent): probability the change does what it
|
||||||
claims. **Break risk** (percent): probability it breaks anything else,
|
claims. **Break risk** (percent): probability it breaks anything else,
|
||||||
@@ -36,16 +33,9 @@ Calibration rules:
|
|||||||
- Static analysis alone caps at the low nineties; only executed evidence
|
- Static analysis alone caps at the low nineties; only executed evidence
|
||||||
(test, reproduction, live probe) justifies more.
|
(test, reproduction, live probe) justifies more.
|
||||||
|
|
||||||
## When a threshold was stated
|
A threshold the operator named ("95% confidence, break risk ≤5%") is a gate on
|
||||||
|
the work, handled as the `break-risk` skill's last section says, for both
|
||||||
|
numbers.
|
||||||
|
|
||||||
A number the operator named ("95% confidence, break risk ≤5%") is a gate on the
|
Objective: the `optimize` skill's. Two figures with their residuals replace a
|
||||||
work, not a field in the report. A block that misses it is not finished output,
|
hedging paragraph and save the operator a verification round trip.
|
||||||
it is a work item: name the cheapest evidence that closes the gap, go get it,
|
|
||||||
re-measure. Report a miss only when that evidence is genuinely out of reach — a
|
|
||||||
deploy you cannot run, an artifact that does not exist — and say which one.
|
|
||||||
|
|
||||||
Objective: the `optimize` skill's. The number exists to save the operator a
|
|
||||||
verification round trip (time), to stop a rework cycle before it starts
|
|
||||||
(quality), and to replace a long hedging paragraph with two figures and their
|
|
||||||
residuals (tokens). A confidence block that costs more than it saves — padding,
|
|
||||||
repeated caveats, a number without residuals — misses its own objective.
|
|
||||||
Reference in new issue
Block a user