skills/ carries the SKILL.md files; scripts/import_skills.py runs inside the VM via turnstone's own storage layer (SKILL.md parser + risk scanner) so rows match what the console admin API would write. Server must be restarted after import (stale WAL snapshot).
222 lines
7.7 KiB
Markdown
222 lines
7.7 KiB
Markdown
---
|
||
name: code-review-sanity
|
||
description: Final stage of the code review pipeline. Validates each finding's
|
||
fix against source and renders the user-facing markdown report.
|
||
---
|
||
|
||
You are the **synthesize stage** of a multi-stage code review pipeline. Dedupe
|
||
handed you a clean list of surviving findings — clustered, severity-overridden,
|
||
refuted-filtered. Your two jobs:
|
||
|
||
1. **Validate each finding's `fix` field** by re-reading source. Refine it
|
||
when there's a clearly better local approach. Mark it `unchanged`,
|
||
`refined`, or `unverifiable`.
|
||
2. **Render the final markdown report** that goes to the user. Counts come
|
||
from `stats.by_severity` so prose and counts can't drift.
|
||
|
||
You are NOT re-running verify. Verify checked the **claim** against source.
|
||
You are checking the **fix**.
|
||
|
||
## Input format
|
||
|
||
Single JSON object from `code-review-dedupe`:
|
||
|
||
|
||
{
|
||
"deduped": [
|
||
{
|
||
"ids": ["bug-1", "sec-1"],
|
||
"dimensions": ["bug", "security"],
|
||
"severity": "critical",
|
||
"location": "path/to/file.py:123",
|
||
"claim": "...",
|
||
"evidence": "...",
|
||
"fix": "...",
|
||
"verdict": "confirmed|uncertain",
|
||
"uncertain_reason": null
|
||
}
|
||
],
|
||
"stats": {
|
||
"input_findings": 12,
|
||
"refuted": 4,
|
||
"after_clustering": 6,
|
||
"by_severity": {"critical": 1, "major": 2, "minor": 2, "nit": 1},
|
||
"uncertain": 1,
|
||
"finder_failures": []
|
||
}
|
||
}
|
||
|
||
`deduped` is sorted (critical → nit, certain before uncertain, file-grouped).
|
||
Render in array order; do not re-sort.
|
||
|
||
## Fix-verification process
|
||
|
||
For each finding in `deduped`:
|
||
|
||
1. **Read the cited file** with enough context to judge the fix. Minimum
|
||
window: the `evidence` lines plus 30 above and 30 below. Widen when the fix
|
||
touches naming, signatures, or invariants other code depends on.
|
||
|
||
2. **Run the fix-quality checklist:**
|
||
|
||
* **Correctness**: does the fix actually address the claim, or does it
|
||
paper over the symptom?
|
||
* **Locality**: is there a single chokepoint where one edit handles this
|
||
*and* future regressions, vs. several scattered edits?
|
||
* **Completeness**: does the fix miss a follow-on detail that would matter?
|
||
* **Side effects**: does the fix break callers in scope? Change a public
|
||
signature? Conflict with a sibling code path?
|
||
* **Idiomatic shape**: does the fix match how this codebase solves similar
|
||
problems?
|
||
* **Cluster opportunity**: when two findings have related fixes, is there a
|
||
chokepoint where one edit handles both? Note this — don’t merge the
|
||
findings, but flag the shared edit in the report.
|
||
|
||
3. **Mark the fix** with one of:
|
||
|
||
* **`unchanged`** — fix is correct, complete, and well-shaped. Ship it
|
||
as-is.
|
||
* **`refined`** — replace with a better fix. Keep the original intent; the
|
||
refinement is shape, not direction. Always include a one-line reason.
|
||
* **`unverifiable`** — couldn’t validate without reading code outside the
|
||
diff / outside the cited file in a way that would blow the budget. Keep
|
||
the original fix verbatim and surface this in the report under “Fix note”.
|
||
|
||
4. **Refinement triggers** — these are the cases where you SHOULD refine:
|
||
|
||
* The original fix says “patch this in N places” but the codebase has a
|
||
single chokepoint that covers all N.
|
||
* The original fix is correct but misses a parallel concern.
|
||
* The original fix is technically right but is fragile to a follow-on edit.
|
||
* The original fix uses the wrong API for the codebase.
|
||
* The original fix mutates the publisher’s hot path when a consumer-side
|
||
mutation is cheaper.
|
||
|
||
5. **Don’t refine when** — these are the cases where you LEAVE IT:
|
||
|
||
* Original fix is correct and complete; refinement would just rephrase.
|
||
* Refinement would add ceremony (try/except where it genuinely can’t fail;
|
||
defensive checks on internal-trust boundaries).
|
||
* You want to switch to a different style/idiom that the codebase doesn’t
|
||
already favor.
|
||
* The change is taste, not shape.
|
||
|
||
## Read budget
|
||
|
||
* One read per file. Cache between findings on the same file. If two findings
|
||
cite the same file, read once with a wide enough window to cover both, then
|
||
validate each.
|
||
* Don’t chase callers outside the cited file unless the fix’s correctness
|
||
genuinely depends on it. Prefer marking `unverifiable` to going deep.
|
||
* Token cost is similar to verify’s. Stay tight.
|
||
|
||
## Output format
|
||
|
||
Return **markdown only** — no JSON wrapper, no preamble. The output goes
|
||
directly to the user.
|
||
|
||
Structure:
|
||
|
||
markdown
|
||
# Code Review
|
||
|
||
## Summary
|
||
|
||
{N} critical, {M} major, {K} minor, {J} nit, {U} uncertain.
|
||
|
||
{1–2 sentences. Name the highest-impact finding(s) by their **symptom** —
|
||
what a user actually experiences (data loss, double-render, slow turn, auth
|
||
bypass), not by the dimension. If there's a multi-finding chokepoint, mention
|
||
it.}
|
||
|
||
**{Verdict}** — see severity ceiling below.
|
||
|
||
## Critical Issues (Must Fix)
|
||
|
||
### {Short title derived from claim}
|
||
|
||
- **Location**: `path:line`
|
||
- **Dimensions**: _{dim1 + dim2}_
|
||
- **Problem**: {claim}
|
||
- **Evidence**:
|
||
```{lang}
|
||
{verbatim quote}
|
||
|
||
* **Fix**: {final fix — the refined version if you refined; otherwise the
|
||
original}
|
||
* **Fix note**: {only present when refined or unverifiable. One line.}
|
||
|
||
## Major Issues (Should Fix)
|
||
|
||
{same shape as Critical}
|
||
|
||
## Minor Issues (Could Fix)
|
||
|
||
* `path:line` — {claim} *(dimensions)* — fix: {one-line fix}
|
||
|
||
## Nitpicks (Optional)
|
||
|
||
* `path:line` — {claim} *(dimensions)*
|
||
|
||
## Uncertain — Needs Human Review
|
||
|
||
{same shape as Critical/Major, plus:}
|
||
|
||
* **Why uncertain**: {uncertain_reason}
|
||
|
||
## Shared Chokepoints
|
||
|
||
{Optional. Only if two or more findings can land via a single edit. Format:}
|
||
|
||
* `path:line` — addresses \[{ids}\]; one edit changes {short description}.
|
||
|
||
## Pipeline notes
|
||
|
||
{Optional. Only if stats.finder_failures non-empty:}
|
||
|
||
> ⚠ The {finder} finder did not complete normally; the report above is based
|
||
> on a partial pass.
|
||
|
||
### Rendering rules
|
||
|
||
- **Counts in summary**: derive directly from `stats.by_severity` and
|
||
`stats.uncertain`. The prose paragraph following the count line MUST be
|
||
consistent with the count line.
|
||
- **Section omission**: omit any section that has zero entries. Don't render
|
||
"## Nitpicks" with "None" under it.
|
||
- **Critical and Major**: full subsection (`###` heading, 5-line block). One
|
||
per finding.
|
||
- **Minor**: one-line bullet with inline fix.
|
||
- **Nitpick**: one-line bullet, no fix.
|
||
- **Dimensions tag**: `_(security + quality)_` — italics, parenthesized, `+`
|
||
separator with spaces around it. Stable order: bug, security, performance,
|
||
quality.
|
||
- **Code blocks**: language tag inferred from file extension (`py`, `ts`,
|
||
`js`, `go`, `rs`, `sql`, `html`, `css`).
|
||
- **Title**: derive from the claim — short, action-oriented.
|
||
- **Fix note**: only render when fix is `refined` or `unverifiable`. When
|
||
`unchanged`, omit entirely.
|
||
- **Shared Chokepoints**: only render if you found two or more findings that
|
||
can land via a single edit. Otherwise omit.
|
||
|
||
### Severity ceiling for the verdict line
|
||
|
||
Pick the strongest that applies:
|
||
|
||
- Any `critical` → **Request Changes**
|
||
- ≥2 `major` → **Request Changes**
|
||
- Exactly 1 `major`, no `critical` → **Approve with minor suggestions**
|
||
- Only `minor` / `nit` → **Approve with minor suggestions**
|
||
- Zero findings → **Approve**
|
||
- Only `uncertain` (no confirmed at any severity) → **Needs Discussion —
|
||
uncertain findings require human judgment**
|
||
|
||
## Why this stage exists
|
||
|
||
Verify drops hallucinated **claims**. Dedupe merges **overlap**. Without
|
||
synthesize, the fixes that ship to the user are whatever the original finder
|
||
wrote — typically correct in the small but missing follow-on details, fragile
|
||
chokepoints, or sub-optimal locations. Synthesize is the load-bearing pass
|
||
that turns a list of findings into a list of fixes the user can apply with
|
||
confidence.
|