~/skills/errata
/errata
QualityA final pre-ship gate that runs your feature against a catalog of the mundane, foreseeable bugs developers routinely forget — sequences, concurrency, stale state, empty/error handling — and returns a PASS or BLOCK verdict with the shortest reproducing case.
The problem
AI coding agents produce finished-looking code fast. Unit tests pass. The diff looks clean. The review looks fine. And then it ships — and someone double-taps a button and charges themselves twice, or tabs away mid-flow and comes back to broken state, or two users edit the same record and one silently loses their work.
These bugs aren’t exotic. They’re mundane — the ones every experienced developer has seen a dozen times. They survive testing because tests look at actions in isolation under clean conditions. They survive code review because reviewers read functions, not sequences. They surface in production because that’s where real users double-click, tab away, and share resources with each other.
The right mental model isn’t “what could a user invent?” — that’s intractable. It’s “which known classes of forgotten behavior does this feature’s surface activate?” That’s a bounded check with a known answer, and it’s exactly what errata runs.
How it works
Phase 1 (Scope) bounds the check to the diff — not the app. It detects the feature’s traits: does it write state? Call something that can fail? Can two actors touch the same resource? Hold state across steps? The trait list is the only input the rest of the check needs.
Phase 2 (Activate) maps traits to catalog classes using each class’s trigger. A read-only screen doesn’t get the double-submit check. A feature with no shared resource doesn’t get the concurrency battery. This is what keeps errata cheap: irrelevant classes never run. The catalog organizes classes into three bands — A (table stakes, usually caught by linters and types), B (frequently forgotten), and C (almost always forgotten). Band C is the prize — interruption, ordering, concurrency, staleness — the territory that unit tests and code review structurally can’t reach.
Phase 3 (Route) splits active classes by engine. Static classes run a bundled Python scanner that detects code smells deterministically over the diff: float money arithmetic, hand-rolled date math, boolean soup async state, swallowed catch blocks, missing empty/error state branches. It emits candidates tagged by catalog class and confidence — high means gate-grade, low means a hint that needs reachability confirmation. Sequence classes (interruption, concurrency, state transitions) go to the opt-in execution mode if requested.
Phase 4 (Oracle gate) filters candidates to findings. A candidate only becomes a finding if there’s a reachable path to trigger it and an oracle that says the result is wrong — a crash/corruption floor, an idempotence violation, an invariant break. No reachable path means the candidate is dropped. This is the precision valve. False positives are the only thing that kills a gate like this, so the gate only fires when it can prove the path.
Phase 5 (Verdict) outputs either PASS or BLOCK. For each blocking finding: the catalog class, a minimized repro (3 steps, not 40), and the oracle that flags it. Nothing else — no padding, no speculation. Each BLOCK repro is exactly the input Sleuth needs to diagnose and fix.
the skill
3 files. Run the installer to set everything up at once, or copy the files individually below.
Invoke with /errata at the start of any relevant task.
#!/usr/bin/env bash
set -e
mkdir -p "$HOME/.claude/skills/errata" "$HOME/.claude/skills/errata/references" "$HOME/.claude/skills/errata/scripts"
cat > "$HOME/.claude/skills/errata/SKILL.md" << '__EOF_0__'
---
name: errata
description: >-
Final pre-launch bug check that runs a feature against a catalog of the mundane, foreseeable
behaviors developers routinely forget — interruptions, concurrency, stale state, empty/error
states, real-world data, permission edges — and returns a pass/block verdict with the shortest
reproducing case. Use this as the LAST gate before shipping a feature, after the obvious bugs are
already fixed: when the user says they're about to ship/launch/merge/release a feature, asks "what
am I missing," "is this ready to ship," "what could break in production," "did I handle all the
edge cases," "give this a final once-over," or "review this before I ship." Trigger even when they
never say "bug" — any pre-launch "is this safe to ship?" moment counts. Cheap by default; do not
skip it because the code looks finished — done-looking code is exactly when forgotten-behavior
bugs hide.
---
# Errata
You are running the **last check before a feature ships**. Your job is to catch the mundane,
foreseeable things real users and real environments do that the author never thought through —
the *sequence × mess* bugs that pass every unit test and every code review, because those look at
actions in isolation under clean conditions, and these only surface once real people hit the
feature in production.
Two failure modes destroy a gate like this, and both end the same way — it stops getting run:
- **Crying wolf.** A false positive teaches the team to click through you. Precision is
everything; a finding you can't tie to a reachable path is worse than silence.
- **Being heavy.** If you need setup, a harness the user must build, or a long wait, you get
skipped — and a skipped gate catches nothing. Stay cheap by default.
A third trap is subtler: **do not try to predict what users will invent.** That is intractable for
everyone, including the author. Your job is not prediction — it is *applying a bounded, known
taxonomy* of forgotten behaviors to the changed surface. The "creativity" of nasty production bugs
lives in the **combinations** of mundane behaviors, never in imagining novel acts.
The core discipline: **check the diff against a catalog, intersect "the code permits it" with "a
human plausibly does it," and report only what's reachable.** The code is ground truth for what's
reachable; you only filter for plausibility. Prefer what the code makes possible over what you
imagine the user intends.
## Operating principles
These hold across all five phases.
- **Precision over completeness.** A gate lives or dies on its false-positive rate. When unsure
whether a finding is real and reachable, drop it. Missing a bug is recoverable; training the
team to ignore you is not.
- **Cheap or skipped.** The default run needs no harness, no user setup, and is scoped to the
diff. If the run starts to feel heavy, you've drifted off the diff or out of static mode —
pull back. The gate that runs every time beats the thorough gate that runs never.
- **Apply the taxonomy, don't predict.** Every check traces to a named class in the catalog.
You are not brainstorming what could go wrong; you are running a known list against this surface.
- **Band C is the prize.** Bands A–B overlap with linters, types, and existing tests — don't burn
budget there. The value is the *sequence / second-actor / time* classes nothing else explores.
- **Reachability-gates everything.** A condition is only a finding if you can show the path is
reachable for this feature. No reachability, no finding.
- **Severity is the user's call.** You deliver a verdict and reproductions; whether a given risk
is acceptable to ship is theirs to decide, not yours to soften or inflate.
## The catalog is the reference core
Your checklist lives in **`references/forgotten-behaviors-catalog.md`**. It is organized by
**forgettability band** (A table-stakes – C almost-always-forgotten); every item is tagged by
**engine** (`[STATIC]` vs `[SEQUENCE]`) and carries an *applies-when* trigger. It also holds the
Oracle Layer and the concrete test corpora.
**Load it selectively — reading all of it every run violates "cheap or skipped."** The catalog
opens with a table of contents. The flow is: detect traits (Phase 1) → from the TOC, open only the
classes those traits activate (Phase 2) → skip Band A unless something cheap flags it. A typical
run reads a handful of class entries plus the Oracle Layer, not the whole file. Do not reconstruct
class definitions from memory — open the specific entries — but do not slurp the entire catalog
either.
The file is the source of truth and it **compounds**: every real production escape becomes a new
entry or a sharpened trigger.
## Maintain a verdict block
Keep a compact running block, updated as you move, and show it to the user.
```
ERRATA STATUS
Mode: <static (default) | +execution>
Scope: <the diff / feature surface under check>
Active classes: <which catalog classes this feature's traits triggered>
Findings: <n reachable> / <n raw>
Verdict: <PASS | BLOCK — see repros | pending>
```
The user may narrow scope or drop a class at any time ("skip the i18n stuff," "don't bother with
concurrency here") — honor it immediately and note it in the block.
---
## Phase 1 — Scope (cheap by construction)
Bound the check to the **change**, not the app. The diff — or the named feature — is the surface;
enumerating the whole codebase is what makes this expensive and is never the job.
Detect the feature's **traits**, the questions that decide which catalog classes are even live:
- Does it read/display data? write or mutate state? hold state across multiple actions?
- Can two actors (tabs, devices, users, requests) touch the same resource?
- Does it call anything that can fail (network, DB, third party, disk)?
- Is it access-controlled? Does it render a data-driven UI? Run in a native shell (mobile)?
- Does it handle real-world data (time, money, names, addresses, i18n, files)?
While here, note the existing **run substrate** (test runner, dev server, type-checker). You need
it only if execution mode is requested; if none exists, static mode still runs fully.
**Exit criterion:** a bounded surface and a trait list.
---
## Phase 2 — Activate (triggers do the scoping)
Map traits → catalog classes using each class's *applies-when* trigger, opening only those entries
from the catalog's TOC. A read-only screen does not get checked for double-submit; a feature with
no shared resource does not get the concurrency battery. This is what keeps the run cheap and the
findings relevant.
Skip Band A unless something cheap flags it — linters and types already hold it. **Weight Band C**
(interruption, ordering, concurrency, partial failure, staleness, auth edges, cross-feature, the
sequence half of UI state). That is the only territory nothing else in the pipeline reaches, and
it is where the budget belongs.
**Exit criterion:** the active class list, Band-C-weighted.
---
## Phase 3 — Route by engine
Each active class is tagged `[STATIC]` or `[SEQUENCE]`. Route accordingly:
- **Static — the default, always-on pass.** Don't re-derive these by hand. Run the bundled
scanner, which detects the `[STATIC]` code smells deterministically over the diff:
```bash
python scripts/scan_static.py # scans working-tree changes
python scripts/scan_static.py --base main # scans everything changed vs a base ref
python scripts/scan_static.py path/a.tsx # scans explicit files
```
It emits candidates tagged by catalog class and `confidence` (`high` → gate-grade,
`medium` = likely-real, `low` = a probably-missing-state hint), and marks whether each sits on a
changed line (`in_diff`). Covered today: float money, hand-rolled date math, boolean-soup /
bag-of-optionals state, permission-checked-once, swallowed catch/`except`, and missing
empty/error states. The scanner is a *candidate generator, not a verdict* — every candidate still
passes through the Phase 4 reachability gate, and `low` candidates in particular need the model
to confirm a state is genuinely missing and reachable. Read the printed snippet rather than
trusting the class label alone. Anything the scanner can't see statically (a smell it has no rule
for) you still inspect by hand — but the scanner is the floor, not the ceiling.
- **Execution — opt-in deep mode.** For `[SEQUENCE]` classes (interruption, concurrency, state
transitions) that can't be judged statically. Requires the substrate found in Phase 1 — **drive
the existing harness; never make the user build one.** Offer it; don't assume it.
Default to static-only unless the user asks for the deep pass or the change is high-blast-radius.
State which mode you're in.
**Exit criterion:** each finding-candidate assigned an engine.
---
## Phase 4 — Check against an oracle, then gate
For each candidate condition you need an **oracle** — how you know the result is *wrong* without a
hand-written spec. Use the cheapest that applies (full detail in the catalog's Oracle Layer):
1. **Crash/corruption floor** (free, near-zero false positives) — an unhandled exception, corrupted
or leaked state, or a contract violation in a plausible flow. No spec needed.
2. **Property oracles** — round-trip, idempotence, invariant, metamorphic.
3. **Differential** (opt-in) — run the condition against the prior version; a divergence the change
didn't intend is a regression.
Then the **reachability gate**, the precision valve: a condition becomes a *finding* only if you
can show the path is actually reachable for this feature. If you can't, drop it — an unreachable
"finding" is the exact false positive that gets the gate ignored. This applies to scanner
candidates too: a `high`-confidence smell on a code path nothing can reach is still a drop.
**Exit criterion:** every surviving finding is reachable and oracle-backed; the rest are dropped.
---
## Phase 5 — Verdict and minimized repro
Output a **verdict, not a report.** **PASS**, or **BLOCK** — and for each blocking finding, the
**shortest action sequence that reproduces it.** Minimize aggressively: a 3-step repro, not the
40-step path you found it on. "Something's wrong somewhere in these flows" is useless; "BLOCK:
submit → background app → reopen → submit → double charge" is the whole product.
Each minimized repro is exactly the input **Sleuth** wants — hand confirmed cases off to it for
diagnosis and fix. *Errata finds and blocks; Sleuth diagnoses.*
For each finding, report only: the **catalog class**, the **reachable repro**, and the **oracle**
that flags it. Nothing else — no padding, no speculation.
**Exit criterion:** PASS, or BLOCK with a minimized, reachable repro per finding.
---
## Worked examples
### A full BLOCK run
Feature under check: a "Claim daily reward" button in an Expo app — taps an endpoint that
increments a server-side balance, then shows the new total.
```
ERRATA STATUS
Mode: static (default)
Scope: ClaimRewardButton.tsx + claimReward() handler (diff of 2 files)
Active classes: B1 (repetition), C3/C7 (concurrent counting), C9b (states)
Findings: 2 reachable / 5 raw
Verdict: BLOCK — see repros
```
Phase 1 traits: writes state, performs an external call that can fail, increments a value that
matters, renders the result. Phase 2 activates B1, C7, B2, C9b (Band A skipped). Phase 3 static
scan returns five candidates:
```
[high] diff ClaimRewardButton.tsx:14 Swallowed error (B2-swallow)
[high] diff claimReward.ts:22 Float money (B6-money)
[medium] diff ClaimRewardButton.tsx:9 Permission checked once (C9e-permonce)
[low] diff ClaimRewardButton.tsx:31 Missing error state (C9b-error)
[low] ctx claimReward.ts:40 Missing empty state (C9b-empty)
```
Two survive Phase 4 as reachable, oracle-backed findings (the other three are handled below and in
the next example):
- **B1 — double-submit.** The button has no in-flight guard, so a fast double-tap fires
`claimReward()` twice before the first returns.
*Oracle:* idempotence — claiming once and claiming twice must leave the same balance; they don't.
*Repro:* `tap Claim → tap Claim again within ~300ms → balance increments twice`.
- **B2 — swallowed failure (scanner B2-swallow, line 14), promoted by reachability.** The `catch {}`
on the claim call means a network failure leaves the optimistic "+1" on screen while the server
never recorded it.
*Oracle:* crash/corruption floor — UI state and server state diverge with no signal.
*Repro:* `airplane mode → tap Claim → balance shows +1 → reopen app → balance unchanged`.
Verdict delivered:
```
BLOCK (2 findings)
1. [B1 repetition] double-claim
submit → submit (within ~300ms) → balance +2 from one reward
oracle: idempotence violated (claim×2 ≠ claim×1)
2. [B2 error path] optimistic update survives a failed write
airplane mode → tap Claim → UI shows +1 → reopen → server has +0
oracle: crash/corruption floor (UI vs server state diverge, silently)
Both repros are ready for Sleuth to diagnose.
```
Note the float-money `high` candidate (line 22) is real but the user owns severity — it's reported
plainly, not inflated into a blocker on its behalf.
### A killed finding (precision in action)
The scanner flagged `[medium] ClaimRewardButton.tsx:9 Permission checked once (C9e-permonce)` — a
camera-permission check in a mount-only effect. It looks like a Band-C foreground-revocation bug.
Reachability gate: the permission is requested at line 9, but nothing on this surface *uses* the
camera — the call is dead code left from a copy-pasted template, and the claim flow never reads the
permission result. No reachable path turns the stale permission into wrong behavior.
**Dropped.** Reported as nothing — not as a softened "you might want to look at the camera
permission." A finding that can't be tied to a reachable consequence is exactly the false positive
that trains the team to click through the gate. Silence here is the correct output. (If you think
the dead call is worth mentioning at all, it's a cleanup note, not an Errata finding — keep it out
of the verdict.)
---
## A note on cost
Errata is meant to be cheap enough that it never gets skipped. The static default needs no harness,
no setup, scopes to the diff, and leans on the bundled scanner for the mechanical checks — it
should feel like a fast lint pass you run on every feature. Execution mode is the deliberate,
opt-in deep dive for high-stakes changes. The structure costs nothing when the feature is simple;
it only does real work where the change actually has the surface for it. Don't pad a run to look
thorough — a clean PASS on a small diff is a complete, correct result.
__EOF_0__
cat > "$HOME/.claude/skills/errata/references/forgotten-behaviors-catalog.md" << '__EOF_1__'
# The Forgotten-Behaviors Catalog
The set of mundane, foreseeable things real users (and real environments) do that developers
routinely fail to think through — and that therefore survive testing to surface in production.
This is the reference the skill checks a feature against.
## How to read this
Two axes run through the catalog:
- **Forgettability band (A / B / C)** — how reliably this class is *already* caught by linters,
types, and existing tests. This routes effort: don't spend budget where Band A already holds.
- **Applies-when trigger** — the feature trait that activates a class. This is what keeps the
skill cheap: a read-only screen never gets checked for double-submit.
The classification borrows the key distinction from Orthogonal Defect Classification: a defect's
*type* (what's wrong in the code) is separate from its *trigger* (the condition that surfaces it).
This catalog is a **trigger taxonomy** — it enumerates the conditions, because those are what a
dev fails to imagine, and what existing tests fail to exercise.
The skill's loop: detect feature traits → activate triggered classes → generate the condition →
check against an oracle (see the Oracle Layer at the end) → report only reachable, real failures.
## Contents (routing table — open only what the traits activate)
This TOC *is* the scoping tool. Match the detected traits to the *applies-when* column and read
only those entries; do not load the whole catalog. `[S]` = STATIC engine (covered by
`scan_static.py`), `[Q]` = SEQUENCE engine (needs execution), `[C]` = a completeness gap.
**Band A — table stakes (skip unless something cheap flags it)**
- `A1` Empty / zero / nothing — *reads or displays any value/collection* `[S]`
- `A2` Basic input shape — *accepts typed or pasted input* `[S]`
- `A3` Boundary / off-by-one — *has any limit, range, or pagination* `[S]`
**Band B — frequently forgotten**
- `B1` Repetition / idempotency — *performs a write, mutation, or external call* `[Q]`
- `B2` Error & failure paths — *calls anything that can fail* `[S]` (swallow) `[Q]` (partial)
- `B3` Lifecycle / cleanup — *acquires a resource, subscription, listener, or lock* `[S/Q]`
- `B4` Volume / scale — *renders/processes collections that grow over time* `[Q]`
- `B5` Config / deployment / environment — *reads config, flags, locale, secrets* `[S]`
- `B6` Real-world data assumptions — *stores/validates/formats/sorts real-world data* `[S]`
(money, time, names, email, addresses, phone, geo, i18n, networks, paths) — pair with corpora
**Band C — almost always forgotten (the prize; weight here)**
- `C1` Interruption / partial completion — *any multi-step flow or in-flight operation* `[Q]`
- `C2` Ordering / sequence — *multiple actions touching shared state* `[Q]`
- `C3` Concurrency (second-actor) — *same resource touched by 2 sessions/devices/users* `[Q]`
- `C4` Distributed / partial failure — *spans services, a queue, any network boundary* `[Q]`
- `C5` State staleness / cache / sync — *caches, mirrors, or derives from a source of truth* `[Q]`
- `C6` Identity / auth / permission edges — *access-controlled or resource-scoped* `[S/Q]`
- `C7` Counting / money / aggregation under concurrency — *counts/sums/decrements* `[Q]`
- `C8` Cross-feature interaction — *shares state/resources/side effects with another feature* `[Q]`
- `C9` UI state completeness — *any data-driven view*; splits by engine:
- `C9a` State stack (ideal/loading/partial/error/empty) `[C]`
- `C9b` Empty/loading/error are not single states `[C]`
- `C9c` Impossible-state code smells (boolean soup, optional bag, perm-once) `[S]`
- `C9d` Combined & transition states `[Q]`
- `C9e` Mobile / React Native lifecycle (permission richness, foreground, cold start) `[S/Q]`
- `C9f` Content & environment variations (type scaling, RTL, dark mode, overflow) `[S/C]`
**Then, regardless of class:** `Oracle Layer` (how you decide a triggered behavior is a bug) and
`Concrete test corpora` (don't hand-roll B6 inputs).
---
## Band A — Table stakes (usually already caught; low marginal value)
Included for completeness so the skill can certify it considered them. Assume linters, type
systems, and existing unit tests cover most of this.
### A1. Empty / zero / nothing
*Applies when:* the feature reads or displays any value or collection.
- Zero items; exactly one item (singular/plural copy; deleting the last one — empty again).
- Zero as a *value* vs null/undefined — the falsy trap where `if (count)` silently skips zero.
- Empty string vs null vs whitespace-only.
- First run / cold start: no data, no history, brand-new account.
- "Had data, now all deleted" — distinct from never-had-data.
### A2. Basic input shape
*Applies when:* the feature accepts typed or pasted input.
- Leading/trailing whitespace; smart quotes; control characters from a paste.
- Negative, zero, or decimal where a positive integer was assumed.
- Numbers-as-strings and type-coercion surprises.
- Characters that break parsing or escaping (`"`, `<`, `&`, null byte).
*(The deeper data-assumption failures live in Band B — "Real-world data".)*
### A3. Boundary / off-by-one
*Applies when:* the feature has any limit, range, or pagination.
- Exactly at the limit; one over; one under.
- Min/max of a range; the limit changing while the user sits at the boundary.
---
## Band B — Frequently forgotten (solid value)
These require imagining the unhappy path, the heavy user, or messy real-world data. Skipped
under time pressure rather than out of ignorance.
### B1. Repetition / idempotency
*Applies when:* the feature performs a write, mutation, or external call.
- Double-click / double-submit before the first call returns.
- Retry after a *perceived* failure that actually succeeded server-side.
- Refresh during an in-flight operation.
- The same request arriving twice (network retry, back-then-forward).
- Creating a duplicate of something meant to be unique.
### B2. Error & failure paths
*Applies when:* the feature calls anything that can fail (network, DB, third party, disk).
- Dependency down, slow, or returning malformed data.
- Partial success (3 of 5 saved) — what state is the user left in?
- The error handler itself throws.
- Silent swallow: a `catch` that hides the failure and continues as if fine.
- Failure *after* a side effect already happened (charged but not recorded).
- Retry storm: failure → retry → more load → more failure (the cascading-failure seed).
### B3. Lifecycle / cleanup
*Applies when:* the feature acquires a resource, subscription, listener, or lock.
- Resource not released on the *error* path (handle, lock, connection, timer).
- Listener/subscription leak across remounts or repeated entries.
- Async resolves *after* the component/context is gone (the "setState after unmount" classic).
### B4. Volume / scale
*Applies when:* the feature renders or processes collections that grow over time.
- The five-year account vs the demo account; the 10,000-item list.
- A very long single string overflowing layout.
- Work that's fine on the dev's three records and quadratic at production size.
- Many rapid actions in succession.
### B5. Configuration / deployment / environment
*Applies when:* the feature reads config, flags, locale, secrets, or env-specific values.
- A feature flag in an unexpected combination with another flag.
- Config consumed without validation (a leading cause of real deploy-time outages).
- A default that's wrong for a real user (locale, timezone, currency, units).
- Works in dev, breaks in prod (config, secrets, real latency, real scale).
- Missing optional config silently treated as present.
### B6. Real-world data assumptions (the "falsehoods" canon)
*Applies when:* the feature stores, validates, formats, sorts, or compares any real-world data.
This is the largest forgotten branch — each item below is a *false assumption* devs encode by
default. Pair this section with concrete corpora (see References) rather than hand-rolled cases.
- **Time & dates.** Days aren't always 24h (DST); a day/week/month can begin and end in
different years; a wall-clock time can occur twice or never (DST transitions); "store
everything in UTC" does not solve future-event scheduling; leap seconds and leap days exist;
the server's timezone is not the user's; timezone *rules themselves* change. Treat any
hand-rolled date logic as a finding by default.
- **Names & identity.** Names aren't ASCII, aren't first/last, can change, can be a single
character or very long, can contain spaces/punctuation, and may not be unique. Gender and
family structure resist enum modeling. (The literal user named "Null" breaks naive systems.)
- **Email.** "Exactly one `@`, validate by regex" is wrong — the RFC permits far more than
intuition allows. Don't validate by regex; send a confirmation.
- **Addresses.** No postal codes in some countries; cities without streets; non-Latin scripts;
addresses that are landmarks, not lines. Regex and addresses don't mix.
- **Phone numbers.** Length, format, and country rules vary wildly; a number isn't a stable
unique key. Use a real library, not a pattern.
- **Geography & coordinates.** Coordinate systems differ; the date line and poles are edge
cases; place names are non-unique and non-Latin; borders and country counts change.
- **Internationalization / Unicode.** String length ≠ grapheme count (combining chars,
surrogate pairs, emoji); case-folding is locale-dependent; normalization forms differ;
substring/truncation can split a grapheme; RTL and bidi reorder display.
- **Money & numbers.** Never use floats for money (`0.1 + 0.2 ≠ 0.3`); use decimals or integer
minor units; mind rounding direction and currency-specific minor-unit counts. Separating
"pennies from dollars" has caused real 100× and $25k errors.
- **Networks / IP.** Multiple valid notations for an IP; IPv6 exists; the Fallacies of
Distributed Computing (the network is *not* reliable, latency *not* zero, bandwidth *not*
infinite, topology *not* static) are all assumptions devs silently bake in.
- **File paths, URLs, CSVs, versions, pagination.** Each has its own canon of broken
assumptions (Windows vs POSIX paths; URL component parsing; CSV quoting/embedded newlines;
version-string ordering; pagination under concurrent insertion/deletion).
---
## Band C — Almost always forgotten (the prize)
Every class here requires imagining one of three things a dev writing a function in isolation
never does: **a sequence of actions**, **a second actor**, or **time passing between steps**.
This is the production-only family, and it is exactly the surface that unit tests and code review
structurally cannot reach. Spend the skill's budget here.
### C1. Interruption / partial completion
*Applies when:* the feature has any multi-step flow or in-flight operation.
- App backgrounded mid-transaction (mobile); OS kills it before resume.
- Network drops mid-operation — what is the recoverable state?
- User navigates away / closes the tab mid-flow.
- Browser back button mid-wizard with state half-applied.
- Session/token expires mid-flow.
- Client-side timeout fires but the operation completed server-side.
### C2. Ordering / sequence (the dev assumed an order)
*Applies when:* the feature has multiple actions touching shared state.
- Actions taken out of the intended order (step 3 before step 1).
- Acting on a view that is now stale (optimistic UI lying about server truth).
- Editing something deleted in another tab or by another path.
- Undo after a dependent action already consumed the thing.
- Two operations racing — which finishes first, and does the code assume?
### C3. Concurrency (the second-actor sub-taxonomy)
*Applies when:* the same resource can be touched by two sessions, devices, users, or requests.
These look correct in isolation and pass most tests; they stay hidden until concurrency rises,
so a feature fine for a handful of users can fail catastrophically as it succeeds and scales.
- **Lost update (read-modify-write).** Two actors read, compute, write back; one silently
overwrites the other. The canonical counter/balance bug.
- **Check-then-act / TOCTOU.** State observed, then acted on assuming it's unchanged — the gap
between check and use is the bug (and a classic security hole: cancel-but-still-paid,
permission-checked-then-request-modified, reserve-then-oversell).
- **Last-write-wins clobber.** Two users edit the same record; one's edit vanishes with no
conflict surfaced.
- **Same user, two contexts.** Two tabs / two devices with diverging local state.
- **Deadlock / livelock.** Circular waits, or threads endlessly yielding without progress.
- **Order/atomicity violation.** A multi-step operation interrupted partway leaves an invalid
intermediate state visible.
### C4. Distributed / partial failure
*Applies when:* the feature spans services, a queue, or any network boundary.
- Partial failure: some calls in a fan-out succeed, some fail — is the whole left consistent?
- Non-idempotent retries double-apply an effect (the retry safety property).
- Cascading failure: one slow dependency exhausts a pool and takes down callers.
- Event-driven assumptions: messages arrive out of order, more than once, or never; "exactly
once" is usually a fiction.
### C5. State staleness / cache / sync
*Applies when:* the feature caches, mirrors, or derives data from a source of truth.
- Cached data shown after the underlying thing changed.
- Optimistic update the server then rejects — does the UI roll back cleanly?
- Two sources of truth quietly disagreeing.
- A permission or flag flip not reflected until reload.
### C6. Identity / auth / permission edges
*Applies when:* the feature is access-controlled or resource-scoped.
- Logged-out user hitting a logged-in path via deep link or bookmark.
- Permissions changed mid-session; the open page still assumes the old ones.
- Accessing another user's resource by changing an ID in the URL (IDOR).
- Account in a weird state: unverified, suspended, trial-expired, mid-migration.
### C7. Counting / money / aggregation under concurrency
*Applies when:* the feature counts, sums, or decrements anything that matters.
- Concurrent decrements overselling inventory / double-spending a credit.
- Aggregate computed over a stale or partial set.
- Negative balance or quantity reached by an unguarded concurrent path.
- Sums that don't reconcile (rounding, silently excluded items).
### C8. Cross-feature interaction
*Applies when:* the feature shares state, resources, or side effects with another feature.
- This feature's state colliding with another's over a shared resource.
- Notifications/emails fired by an action the user then undid.
- A data migration this feature didn't know about.
- A side effect here that another feature's invariant silently depends on.
### C9. UI state completeness (frontend) — expanded section
The richest forgotten zone for any UI surface, and it splits cleanly along the static-vs-execute
line: some sub-classes are **[STATIC]** code smells detectable without running anything, others
are **[SEQUENCE]** bugs that need execution, and others are **[COMPLETENESS]** gaps (a state the
designer never drew). Each item below is tagged so the skill routes it to the right engine.
Why weight this class heavily, especially for AI-written code: forgotten states are
underrepresented in training data, so AI-generated frontends skip them at unusually high rates —
one 2025 analysis of AI-built dashboards found near-total absence of empty and error states and a
universally lazy loading state, attributed to training data being dominated by happy-path
tutorials rather than production edge handling.
#### C9a. State completeness — the UI Stack [COMPLETENESS]
*Applies when:* the feature is any data-driven view.
The canonical five states are **ideal, loading, partial, error, empty** — a UI missing any of
them feels broken. The two most-skipped:
- **Partial:** sparse/incomplete data (one item where hundreds were assumed; some fields present,
others missing) so features like pagination never activate and layout collapses.
- Each of the five fragments further (below), and the fragments are where bugs actually live.
#### C9b. Empty / loading / error are not single states [COMPLETENESS]
*Applies when:* the view fetches or mutates data.
- **Empty is ≥4 distinct states**, each with different copy and correct action: first-run /
onboarding (never had data), no-search-results (query matched nothing), user-cleared (deleted
everything), no-permission, and the bug-prone *error-rendered-as-empty* (failed load shown as
"nothing here").
- **Loading is several:** initial load (skeleton), refetch-with-stale-data-visible, load-more /
pagination, background refresh. Sub-bugs: flash-of-spinner on a sub-100ms load; spinner with no
timeout; discarding good data to show a spinner on refetch.
- **Error is several:** page-level vs section-level vs field-level; retryable vs fatal; validation
vs system; partial-batch failure; error-with-last-valid-data (keep showing the last good value).
#### C9c. Impossible-state code smells [STATIC]
*Applies when:* component/store models one async resource. **Detectable without execution** — this
is the cause of "spinner AND error at once" rendering bugs, and a high-precision static finding.
- **Boolean soup:** multiple correlated flags (`isLoading`, `isError`, `isSuccess`) modeling one
resource create 2^n states, most invalid; nothing prevents `isLoading && isError`.
- **Bag of optionals:** `{ loading?, data?, error? }` permits success-without-data and
loading-carrying-an-error.
- **Permission/state checked once at mount** and trusted thereafter (see C9e mobile).
- Healthy form (the suggested fix, not a bug): a discriminated union / state machine where
invalid combinations don't compile — and which can model the *legitimate* combined states
explicitly (`loading` with `staleData`, `success` with `isStale`, `error` with `lastValidData`).
#### C9d. Combined & transition states [SEQUENCE]
*Applies when:* the view moves between states over time.
- Combined: loading-while-showing-stale, success-but-stale, error-but-have-cached-data.
- Transitions: ideal → empty (deleted the last item), success → error (a refetch fails after data
was shown), rapid back-and-forth between states. These need execution to surface.
#### C9e. Mobile / React Native lifecycle states [SEQUENCE / STATIC mix]
*Applies when:* the surface runs in a native app shell (e.g. Expo/RN).
- **Permission richness [STATIC for "checked once", SEQUENCE for revocation]:** the real status
set is granted / denied / **blocked** (permanently denied — can't re-prompt, must route to
Settings) / **unavailable** (not on device) / restricted / undetermined. A one-time grant is
auto-revoked within ~30–60s of the app being terminated or backgrounded-and-unused, and users
can toggle permissions in Settings while backgrounded — so a permission checked only at mount
and trusted afterward is a latent bug; re-check on foreground via AppState.
- App backgrounded/foregrounded mid-flow (auth or data changed while away).
- Deep-link or push-notification **cold start** landing directly on a deep screen with no nav
stack and no warm session.
- Keyboard covering the input or submit button; orientation change mid-flow; safe-area/notch.
- Offline → online transitions; airplane mode mid-action; slow network.
- OTA-update / app-version skew mid-session.
#### C9f. Content & environment variations [COMPLETENESS / STATIC]
*Applies when:* the surface renders text, images, or themed UI.
- Dynamic type / font scaling; RTL layout; dark mode or a system-theme flip mid-session.
- Very long content and truncation; broken/missing images; localized text length overflowing
layout (e.g. German); zero/one/many pluralization copy.
---
## The Oracle Layer — deciding a triggered behavior is actually a bug
Generating a forgotten condition is half the job; the other half is knowing the result is wrong
*without* a hand-written spec. Use these oracles, cheapest first:
1. **Crash / corruption floor (free, near-zero false positives).** Did it throw an unhandled
exception or rejection, corrupt state, leak state between sequences, or violate its own type
contract? No spec needed — an exception in a plausible user flow is essentially never
intended. This catches most of Band C with no user input.
2. **Property oracles (partial oracles — no full spec required).** A property defines a
relationship that must hold, sidestepping the "what's the exact correct output?" problem:
- **Round-trip:** `decode(encode(x)) == x` (serialization, parse/format, save/load).
- **Idempotence:** doing it twice equals doing it once (retries, normalization, dedupe,
`set` operations) — directly validates B1 and C4.
- **Invariant:** something preserved across an operation (collection size after a map, total
after a transfer, balance never negative) — directly validates C7.
- **Metamorphic:** a known input change implies a known output change, when you can't compute
the right answer directly (reorder input → same sorted output; sub-path of a shortest path
is itself shortest). Powerful for logic with no easy reference answer.
3. **Differential oracle (opt-in, higher cost).** Run the condition against the prior version;
any divergence the change wasn't meant to cause is a regression. The old version is the
oracle for everything the feature didn't intend to alter. Note the known weakness: an
independent *reimplementation* tends to share faults on the same inputs, so prefer
prior-version diffing or a deliberately-simple reference, not a parallel rewrite.
Mapping: Band A/B input classes → round-trip + invariant. Idempotency (B1) → idempotence.
Counting/money (C7) → invariant. Anything without a clean property → crash floor + differential.
---
## Concrete test corpora (don't hand-roll these)
For the data classes in B6, feed real corpora rather than inventing cases:
- **Big List of Naughty Strings** — inputs with a high probability of breaking input handling.
- **i18n / international name test data** — real diverse names and scripts.
- **libphonenumber / libaddressinput** — reference validation for phones and addresses.
- The "falsehoods programmers believe" articles per domain — use as the assumption checklist
behind each B6 bullet.
---
## Notes for turning this into the skill
- **Triggers do the scoping.** Feature-trait detection up front (reads? writes? holds state
across actions? multi-user? calls a dependency? renders data?) maps traits → active classes.
This is what keeps it cheap and stops irrelevant checks.
- **Band C is the differentiator.** Bands A–B overlap with linters, types, and existing tests.
A version shipping only Band C + the oracle layer would still beat the entire generic-review
tier, because nothing else explores sequence / second-actor / time.
- **Every check needs a reachability gate.** A class fires as a finding only if the path is
actually reachable for this feature — otherwise you reintroduce the false-positive fatigue
that gets gates ignored.
- **Route by engine, not just by class.** Many classes are mixed: tag each check `[STATIC]`
(a code smell detectable without running — e.g. boolean-soup state, permission-checked-once,
float money, hand-rolled date math) vs `[SEQUENCE]` (needs execution — interruption,
concurrency, state transitions). The static-tagged checks are the cheap always-on default;
the sequence-tagged ones are the opt-in execution mode. C9 is the worked example of this split.
- **The AI-generated-code angle is a wedge.** Forgotten behaviors are, by definition,
underrepresented in training data, so AI-written features skip them at unusually high rates.
A gate aimed at exactly this gap is most valuable precisely where code is AI-authored.
- **Version the catalog.** Every real production escape becomes a new entry or a sharpened
trigger. This is the asset that compounds — and the same loop ODC was built around: turn the
defect stream into a measurement that improves the process.
__EOF_1__
cat > "$HOME/.claude/skills/errata/scripts/scan_static.py" << '__EOF_2__'
#!/usr/bin/env python3
"""
scan_static.py — Errata's static engine.
Runs the deterministic, [STATIC]-tagged checks from the forgotten-behaviors
catalog against a changed surface, so the model does not re-derive them by hand
on every run. Pure stdlib: nothing to install, nothing to skip.
Design ethos matches the skill: PRECISION OVER COMPLETENESS. Every detector is
tuned to rarely false-positive. Findings carry a confidence:
high — gate-grade; a true positive almost every time it fires.
medium — very likely real, but worth a glance.
low — a hint a state is probably missing; the model must confirm
reachability before it ever becomes a finding.
The script never decides severity and never emits a verdict. It produces
*candidates* tagged by catalog class. Phase 4's reachability gate still runs.
Usage:
python scripts/scan_static.py # scan working-tree changes (git diff)
python scripts/scan_static.py --base main # scan everything changed vs `main`
python scripts/scan_static.py src/a.tsx b.ts # scan explicit files
python scripts/scan_static.py --json # JSON only (default prints both)
Exit code is always 0 — this is a candidate generator, not a CI gate. The
verdict belongs to the model and the user.
"""
from __future__ import annotations
import argparse
import json
import os
import re
import subprocess
import sys
from dataclasses import dataclass, asdict, field
from typing import Iterable
JS_EXT = {".js", ".jsx", ".ts", ".tsx", ".mjs", ".cjs"}
PY_EXT = {".py"}
SCANNABLE = JS_EXT | PY_EXT
MAX_BYTES = 2_000_000 # skip generated/minified monsters; they aren't the diff
@dataclass
class Finding:
catalog_id: str
klass: str
file: str
line: int
snippet: str
confidence: str # high | medium | low
why: str
in_diff: bool = False
engine: str = "STATIC"
def to_dict(self) -> dict:
d = asdict(self)
d["class"] = d.pop("klass")
return d
# --------------------------------------------------------------------------- #
# git plumbing — scope to the change, and learn which lines are actually new. #
# --------------------------------------------------------------------------- #
def _git(args: list[str]) -> str | None:
try:
out = subprocess.run(
["git", *args],
capture_output=True, text=True, check=True,
)
return out.stdout
except (subprocess.CalledProcessError, FileNotFoundError):
return None
def changed_files(base: str | None) -> list[str]:
spec = [f"{base}...HEAD"] if base else []
out = _git(["diff", "--name-only", *spec, "--diff-filter=d"]) # d = exclude deletes
if out is None:
return []
files = [f.strip() for f in out.splitlines() if f.strip()]
if not files and base is None:
# nothing staged/unstaged differs from HEAD; fall back to last commit
out = _git(["diff", "--name-only", "HEAD~1...HEAD", "--diff-filter=d"])
files = [f.strip() for f in out.splitlines() if f.strip()] if out else []
return files
def added_line_ranges(base: str | None) -> dict[str, list[tuple[int, int]]]:
"""Map file -> list of (start, end) line ranges that the diff adds."""
spec = [f"{base}...HEAD"] if base else []
out = _git(["diff", "-U0", *spec])
ranges: dict[str, list[tuple[int, int]]] = {}
if out is None:
return ranges
cur: str | None = None
hunk = re.compile(r"^@@ -\d+(?:,\d+)? \+(\d+)(?:,(\d+))? @@")
for line in out.splitlines():
if line.startswith("+++ b/"):
cur = line[6:].strip()
ranges.setdefault(cur, [])
elif line.startswith("@@") and cur is not None:
m = hunk.match(line)
if m:
start = int(m.group(1))
count = int(m.group(2)) if m.group(2) else 1
if count > 0:
ranges[cur].append((start, start + count - 1))
return ranges
def in_ranges(line_no: int, ranges: list[tuple[int, int]]) -> bool:
return any(a <= line_no <= b for a, b in ranges)
# --------------------------------------------------------------------------- #
# helpers #
# --------------------------------------------------------------------------- #
def line_of(text: str, idx: int) -> int:
return text.count("\n", 0, idx) + 1
def snippet_at(lines: list[str], line_no: int) -> str:
return lines[line_no - 1].strip()[:160] if 0 < line_no <= len(lines) else ""
def looks_rn(text: str) -> bool:
return bool(re.search(r"from\s+['\"](react-native|expo[\w/-]*)['\"]", text))
MONEY = re.compile(
r"\b\w*(price|amount|total|subtotal|cost|balance|fee|tax|vat|discount|"
r"salary|wage|payroll|payment|charge|refund|credit|debit|cents|dollars|"
r"usd|eur|gbp|money|currenc|invoice|payout|deposit)\w*\b",
re.IGNORECASE,
)
DECIMAL = re.compile(r"\b\d+\.\d+\b")
ARITH = re.compile(r"[-+*/]")
# --------------------------------------------------------------------------- #
# detectors #
# --------------------------------------------------------------------------- #
def detect_float_money(path: str, text: str, lines: list[str]) -> Iterable[Finding]:
for i, ln in enumerate(lines, 1):
code = ln.split("//", 1)[0]
if not MONEY.search(code):
continue
# parseFloat / Number on something money-shaped: high.
if re.search(r"parseFloat\s*\(", code) or re.search(r"\bNumber\s*\(", code):
yield Finding(
"B6-money", "Float money", path, i, ln.strip()[:160], "high",
"parseFloat/Number used on a money-named value — floats can't "
"represent currency exactly (0.1 + 0.2 != 0.3). Use integer "
"minor units or a decimal type.",
)
continue
# money name + decimal literal + arithmetic on the same line: high.
if DECIMAL.search(code) and ARITH.search(code):
yield Finding(
"B6-money", "Float money", path, i, ln.strip()[:160], "high",
"Arithmetic on a money-named value with a float literal. Money "
"in floats accumulates rounding error; use minor units.",
)
continue
# cents<->dollars toggling by *100 / /100: medium (the $25k-error smell).
if re.search(r"[*/]\s*100\b", code):
yield Finding(
"B6-money", "Float money", path, i, ln.strip()[:160], "medium",
"Manual *100 / /100 on a money value — the classic "
"pennies<->dollars conversion that has caused real 100x errors. "
"Confirm a single, tested money type owns this.",
)
# toFixed used to 'round' money: medium.
elif re.search(r"\.toFixed\s*\(\s*2\s*\)", code):
yield Finding(
"B6-money", "Float money", path, i, ln.strip()[:160], "medium",
"toFixed(2) formats but does not round money correctly and "
"still operates on a float. Round in minor units before display.",
)
DATE_MS = re.compile(
r"(24\s*\*\s*60\s*\*\s*60\s*\*\s*1000" # ms per day, common orderings
r"|60\s*\*\s*60\s*\*\s*24\s*\*\s*1000"
r"|1000\s*\*\s*60\s*\*\s*60\s*\*\s*24"
r"|\b86_?400_?000\b" # ms per day literal
r"|\b86_?400\b\s*\*\s*1000)"
)
DATE_COMPONENT_MATH = re.compile(
r"\.(setDate|setMonth|setFullYear|setHours)\s*\(\s*[^)]*\.(getDate|getMonth|"
r"getFullYear|getHours)\s*\(\)\s*[-+]"
)
DATE_EPOCH_MATH = re.compile(r"(getTime\s*\(\)|Date\.now\s*\(\))\s*[-+]")
def detect_handrolled_date(path: str, text: str, lines: list[str]) -> Iterable[Finding]:
for i, ln in enumerate(lines, 1):
code = ln.split("//", 1)[0]
if DATE_MS.search(code):
yield Finding(
"B6-time", "Hand-rolled date math", path, i, ln.strip()[:160], "high",
"Manual milliseconds-per-day arithmetic. Days aren't always 24h "
"(DST), so adding 86,400,000ms skips/repeats an hour twice a year. "
"Use a date library's day arithmetic.",
)
elif DATE_COMPONENT_MATH.search(code):
yield Finding(
"B6-time", "Hand-rolled date math", path, i, ln.strip()[:160], "high",
"Hand-rolled date-component arithmetic. Month/day rollover and "
"DST make this wrong at boundaries; use a library.",
)
elif DATE_EPOCH_MATH.search(code) and re.search(r"day|hour|week|month|expire|ttl|age", code, re.I):
yield Finding(
"B6-time", "Hand-rolled date math", path, i, ln.strip()[:160], "medium",
"Epoch arithmetic used to derive a duration/expiry. Verify it "
"isn't assuming fixed-length days.",
)
# correlated async-resource flags — boolean soup (C9c)
USESTATE = re.compile(r"\b(?:const|let)\s*\[\s*(\w+)\s*,", re.IGNORECASE)
USESTATE_CALL = re.compile(r"=\s*useState\b")
LOADING = re.compile(r"^(is)?(loading|fetching|pending|busy)$", re.I)
ERRORF = re.compile(r"^(is)?error$|^err$|^hasError$", re.I)
SUCCESS = re.compile(r"^(is)?(success|succeeded|loaded|done|ready)$", re.I)
def detect_boolean_soup(path: str, text: str, lines: list[str]) -> Iterable[Finding]:
if os.path.splitext(path)[1] not in JS_EXT:
return
# Collect useState boolean-ish flags with their line numbers.
flags: list[tuple[int, str, str]] = [] # (line, name, concept)
for i, ln in enumerate(lines, 1):
if not USESTATE_CALL.search(ln):
continue
m = USESTATE.search(ln)
if not m:
continue
name = m.group(1)
if LOADING.match(name):
flags.append((i, name, "loading"))
elif ERRORF.match(name):
flags.append((i, name, "error"))
elif SUCCESS.match(name):
flags.append((i, name, "success"))
# Cluster flags within ~40 lines (a rough same-component window).
flags.sort()
n = len(flags)
for a in range(n):
window = [flags[a]]
for b in range(a + 1, n):
if flags[b][0] - flags[a][0] <= 40:
window.append(flags[b])
concepts = {c for _, _, c in window}
if len(concepts) >= 2:
names = ", ".join(sorted({nm for _, nm, _ in window}))
yield Finding(
"C9c-boolsoup", "Impossible-state boolean soup", path, flags[a][0],
snippet_at(lines, flags[a][0]), "high",
f"Separate useState flags ({names}) model one async resource. "
"2^n combinations exist, most invalid — nothing prevents "
"isLoading && isError rendering at once. Model it as one "
"discriminated union / state machine.",
)
break # one finding per component cluster is enough
OPTIONAL_BAG = re.compile(
r"\{\s*(?=[^}]*\bdata\s*\?)(?=[^}]*\b(loading|isLoading)\s*\?)"
r"(?=[^}]*\b(error|isError)\s*\?)[^}]*\}"
)
def detect_optional_bag(path: str, text: str, lines: list[str]) -> Iterable[Finding]:
if os.path.splitext(path)[1] not in JS_EXT:
return
for m in OPTIONAL_BAG.finditer(text):
i = line_of(text, m.start())
yield Finding(
"C9c-optbag", "Bag of optionals", path, i, snippet_at(lines, i), "medium",
"A { data?, loading?, error? } shape permits success-without-data "
"and loading-carrying-an-error. Prefer a discriminated union.",
)
PERM_API = re.compile(
r"(request\w*Permission|get\w*Permission|check\w*Permission|hasPermission|"
r"Permissions\.\w+|requestPermissionsAsync|getPermissionsAsync)",
re.IGNORECASE,
)
MOUNT_EFFECT = re.compile(r"useEffect\s*\([^,]*,\s*\[\s*\]\s*\)", re.DOTALL)
def detect_permission_once(path: str, text: str, lines: list[str]) -> Iterable[Finding]:
if os.path.splitext(path)[1] not in JS_EXT or not looks_rn(text):
return
if not PERM_API.search(text):
return
if "AppState" in text:
return # likely re-checks on foreground — don't cry wolf
if not MOUNT_EFFECT.search(text):
return
m = PERM_API.search(text)
i = line_of(text, m.start())
yield Finding(
"C9e-permonce", "Permission checked once", path, i, snippet_at(lines, i),
"medium",
"Permission is checked in a mount-only effect with no AppState listener. "
"A one-time grant auto-revokes ~30-60s after backgrounding, and users "
"can flip it in Settings while away. Re-check on foreground.",
)
EMPTY_CATCH_JS = re.compile(r"catch\s*(\([^)]*\))?\s*\{\s*\}")
EMPTY_CATCH_ARROW = re.compile(r"\.catch\s*\(\s*\(?[^)]*\)?\s*=>\s*\{\s*\}\s*\)")
LOGONLY_CATCH = re.compile(
r"catch\s*(\([^)]*\))?\s*\{\s*console\.\w+\([^;]*\);?\s*\}"
)
def detect_swallowed_catch_js(path: str, text: str, lines: list[str]) -> Iterable[Finding]:
for m in EMPTY_CATCH_JS.finditer(text):
i = line_of(text, m.start())
yield Finding(
"B2-swallow", "Swallowed error", path, i, snippet_at(lines, i), "high",
"Empty catch block hides the failure and continues as if fine. The "
"user is left in an undefined state with no signal.",
)
for m in EMPTY_CATCH_ARROW.finditer(text):
i = line_of(text, m.start())
yield Finding(
"B2-swallow", "Swallowed error", path, i, snippet_at(lines, i), "high",
"Empty .catch() swallows a rejected promise — the failure path is "
"silently a no-op.",
)
for m in LOGONLY_CATCH.finditer(text):
i = line_of(text, m.start())
yield Finding(
"B2-swallow", "Swallowed error", path, i, snippet_at(lines, i), "medium",
"catch only logs and continues — no recovery, no user-visible error, "
"no rethrow. Decide the real failure state.",
)
BARE_EXCEPT = re.compile(r"except\s*:\s*\n\s*pass", re.MULTILINE)
TYPED_EXCEPT_PASS = re.compile(r"except\s+[\w.]+(?:\s+as\s+\w+)?\s*:\s*\n\s*pass", re.MULTILINE)
def detect_swallowed_except_py(path: str, text: str, lines: list[str]) -> Iterable[Finding]:
if os.path.splitext(path)[1] not in PY_EXT:
return
for rx, conf in ((BARE_EXCEPT, "high"), (TYPED_EXCEPT_PASS, "high")):
for m in rx.finditer(text):
i = line_of(text, m.start())
yield Finding(
"B2-swallow", "Swallowed error", path, i, snippet_at(lines, i), conf,
"except: pass swallows the failure and continues. At minimum log "
"and decide the recovery state.",
)
MAP_RENDER = re.compile(r"\.map\s*\(")
FETCHES = re.compile(r"(useQuery|useSWR|fetch\s*\(|axios|useEffect)", re.I)
# Structural signals only — never prose/comments, which produce both false
# positives ("// handle the empty case later") and false negatives.
HAS_EMPTY = re.compile(r"(\.length\s*===?\s*0|\.length\s*<\s*1|\bisEmpty\b|<Empty|EmptyState)")
HAS_ERROR_UI = re.compile(r"(\bisError\b|\bhasError\b|<Error|ErrorBoundary|ErrorState)")
def _strip_comments(text: str) -> str:
# Replace comments with blanks but keep newline count intact, so line
# numbers computed against the stripped text still match the real file.
text = re.sub(r"/\*.*?\*/", lambda m: "\n" * m.group(0).count("\n"),
text, flags=re.DOTALL)
text = re.sub(r"//[^\n]*", "", text)
return text
def detect_missing_states(path: str, text: str, lines: list[str]) -> Iterable[Finding]:
if os.path.splitext(path)[1] not in JS_EXT:
return
code = _strip_comments(text)
if not (MAP_RENDER.search(code) and FETCHES.search(code)):
return
text = code # check empty/error signals against comment-free source
m = MAP_RENDER.search(text)
i = line_of(text, m.start())
if not HAS_EMPTY.search(text):
yield Finding(
"C9b-empty", "Missing empty state", path, i, snippet_at(lines, i), "low",
"Renders fetched data via .map but no empty-state branch is visible. "
"Empty is >=4 distinct states (first-run, no-results, user-cleared, "
"error-as-empty). Confirm one is handled and reachable.",
)
if not HAS_ERROR_UI.search(text):
yield Finding(
"C9b-error", "Missing error state", path, i, snippet_at(lines, i), "low",
"Fetches and renders but no error branch is visible. A failed load "
"likely renders as a permanent empty/blank screen.",
)
DETECTORS = [
detect_float_money,
detect_handrolled_date,
detect_boolean_soup,
detect_optional_bag,
detect_permission_once,
detect_swallowed_catch_js,
detect_swallowed_except_py,
detect_missing_states,
]
# --------------------------------------------------------------------------- #
# driver #
# --------------------------------------------------------------------------- #
def scan_file(path: str) -> list[Finding]:
try:
if os.path.getsize(path) > MAX_BYTES:
return []
with open(path, "r", encoding="utf-8", errors="replace") as fh:
text = fh.read()
except (OSError, ValueError):
return []
lines = text.splitlines()
out: list[Finding] = []
for det in DETECTORS:
try:
out.extend(det(path, text, lines))
except re.error:
continue
return out
def resolve_targets(args) -> tuple[list[str], str]:
if args.paths:
return [p for p in args.paths if os.path.splitext(p)[1] in SCANNABLE], "paths"
files = changed_files(args.base)
files = [f for f in files if os.path.splitext(f)[1] in SCANNABLE and os.path.exists(f)]
return files, ("git-diff" if not args.base else f"git-diff:{args.base}")
def main() -> int:
ap = argparse.ArgumentParser(description="Errata static engine.")
ap.add_argument("paths", nargs="*", help="explicit files to scan")
ap.add_argument("--base", help="git ref to diff against (default: working-tree changes)")
ap.add_argument("--json", action="store_true", help="emit JSON only")
args = ap.parse_args()
targets, mode = resolve_targets(args)
ranges = added_line_ranges(args.base) if mode.startswith("git-diff") else {}
findings: list[Finding] = []
for path in targets:
for f in scan_file(path):
f.in_diff = in_ranges(f.line, ranges.get(path, [])) if ranges else True
findings.append(f)
# Diff-touching findings first, then by confidence.
rank = {"high": 0, "medium": 1, "low": 2}
findings.sort(key=lambda f: (not f.in_diff, rank.get(f.confidence, 9), f.file, f.line))
by_conf = {"high": 0, "medium": 0, "low": 0}
by_class: dict[str, int] = {}
for f in findings:
by_conf[f.confidence] = by_conf.get(f.confidence, 0) + 1
by_class[f.klass] = by_class.get(f.klass, 0) + 1
report = {
"scope": {"mode": mode, "files_scanned": len(targets), "files": targets},
"summary": {"by_confidence": by_conf, "by_class": by_class, "total": len(findings)},
"findings": [f.to_dict() for f in findings],
}
if args.json:
print(json.dumps(report, indent=2))
return 0
# Human-readable to stderr, JSON to stdout — easy to pipe or read.
if not targets:
print("scan_static: no scannable changed files found "
"(pass paths explicitly, or check you're in a git repo).", file=sys.stderr)
else:
print(f"scan_static: {len(targets)} file(s), {len(findings)} candidate(s) "
f"[{by_conf['high']} high / {by_conf['medium']} medium / {by_conf['low']} low]",
file=sys.stderr)
for f in findings:
tag = "diff" if f.in_diff else "ctx "
print(f" [{f.confidence:<6}] {tag} {f.file}:{f.line} {f.klass} ({f.catalog_id})",
file=sys.stderr)
print(json.dumps(report, indent=2))
return 0
if __name__ == "__main__":
sys.exit(main())
__EOF_2__
chmod +x "$HOME/.claude/skills/errata/scripts/scan_static.py"
echo "✓ errata installed → ~/.claude/skills/errata/"~/.claude/skills/errata/SKILL.md---
name: errata
description: >-
Final pre-launch bug check that runs a feature against a catalog of the mundane, foreseeable
behaviors developers routinely forget — interruptions, concurrency, stale state, empty/error
states, real-world data, permission edges — and returns a pass/block verdict with the shortest
reproducing case. Use this as the LAST gate before shipping a feature, after the obvious bugs are
already fixed: when the user says they're about to ship/launch/merge/release a feature, asks "what
am I missing," "is this ready to ship," "what could break in production," "did I handle all the
edge cases," "give this a final once-over," or "review this before I ship." Trigger even when they
never say "bug" — any pre-launch "is this safe to ship?" moment counts. Cheap by default; do not
skip it because the code looks finished — done-looking code is exactly when forgotten-behavior
bugs hide.
---
# Errata
You are running the **last check before a feature ships**. Your job is to catch the mundane,
foreseeable things real users and real environments do that the author never thought through —
the *sequence × mess* bugs that pass every unit test and every code review, because those look at
actions in isolation under clean conditions, and these only surface once real people hit the
feature in production.
Two failure modes destroy a gate like this, and both end the same way — it stops getting run:
- **Crying wolf.** A false positive teaches the team to click through you. Precision is
everything; a finding you can't tie to a reachable path is worse than silence.
- **Being heavy.** If you need setup, a harness the user must build, or a long wait, you get
skipped — and a skipped gate catches nothing. Stay cheap by default.
A third trap is subtler: **do not try to predict what users will invent.** That is intractable for
everyone, including the author. Your job is not prediction — it is *applying a bounded, known
taxonomy* of forgotten behaviors to the changed surface. The "creativity" of nasty production bugs
lives in the **combinations** of mundane behaviors, never in imagining novel acts.
The core discipline: **check the diff against a catalog, intersect "the code permits it" with "a
human plausibly does it," and report only what's reachable.** The code is ground truth for what's
reachable; you only filter for plausibility. Prefer what the code makes possible over what you
imagine the user intends.
## Operating principles
These hold across all five phases.
- **Precision over completeness.** A gate lives or dies on its false-positive rate. When unsure
whether a finding is real and reachable, drop it. Missing a bug is recoverable; training the
team to ignore you is not.
- **Cheap or skipped.** The default run needs no harness, no user setup, and is scoped to the
diff. If the run starts to feel heavy, you've drifted off the diff or out of static mode —
pull back. The gate that runs every time beats the thorough gate that runs never.
- **Apply the taxonomy, don't predict.** Every check traces to a named class in the catalog.
You are not brainstorming what could go wrong; you are running a known list against this surface.
- **Band C is the prize.** Bands A–B overlap with linters, types, and existing tests — don't burn
budget there. The value is the *sequence / second-actor / time* classes nothing else explores.
- **Reachability-gates everything.** A condition is only a finding if you can show the path is
reachable for this feature. No reachability, no finding.
- **Severity is the user's call.** You deliver a verdict and reproductions; whether a given risk
is acceptable to ship is theirs to decide, not yours to soften or inflate.
## The catalog is the reference core
Your checklist lives in **`references/forgotten-behaviors-catalog.md`**. It is organized by
**forgettability band** (A table-stakes – C almost-always-forgotten); every item is tagged by
**engine** (`[STATIC]` vs `[SEQUENCE]`) and carries an *applies-when* trigger. It also holds the
Oracle Layer and the concrete test corpora.
**Load it selectively — reading all of it every run violates "cheap or skipped."** The catalog
opens with a table of contents. The flow is: detect traits (Phase 1) → from the TOC, open only the
classes those traits activate (Phase 2) → skip Band A unless something cheap flags it. A typical
run reads a handful of class entries plus the Oracle Layer, not the whole file. Do not reconstruct
class definitions from memory — open the specific entries — but do not slurp the entire catalog
either.
The file is the source of truth and it **compounds**: every real production escape becomes a new
entry or a sharpened trigger.
## Maintain a verdict block
Keep a compact running block, updated as you move, and show it to the user.
```
ERRATA STATUS
Mode: <static (default) | +execution>
Scope: <the diff / feature surface under check>
Active classes: <which catalog classes this feature's traits triggered>
Findings: <n reachable> / <n raw>
Verdict: <PASS | BLOCK — see repros | pending>
```
The user may narrow scope or drop a class at any time ("skip the i18n stuff," "don't bother with
concurrency here") — honor it immediately and note it in the block.
---
## Phase 1 — Scope (cheap by construction)
Bound the check to the **change**, not the app. The diff — or the named feature — is the surface;
enumerating the whole codebase is what makes this expensive and is never the job.
Detect the feature's **traits**, the questions that decide which catalog classes are even live:
- Does it read/display data? write or mutate state? hold state across multiple actions?
- Can two actors (tabs, devices, users, requests) touch the same resource?
- Does it call anything that can fail (network, DB, third party, disk)?
- Is it access-controlled? Does it render a data-driven UI? Run in a native shell (mobile)?
- Does it handle real-world data (time, money, names, addresses, i18n, files)?
While here, note the existing **run substrate** (test runner, dev server, type-checker). You need
it only if execution mode is requested; if none exists, static mode still runs fully.
**Exit criterion:** a bounded surface and a trait list.
---
## Phase 2 — Activate (triggers do the scoping)
Map traits → catalog classes using each class's *applies-when* trigger, opening only those entries
from the catalog's TOC. A read-only screen does not get checked for double-submit; a feature with
no shared resource does not get the concurrency battery. This is what keeps the run cheap and the
findings relevant.
Skip Band A unless something cheap flags it — linters and types already hold it. **Weight Band C**
(interruption, ordering, concurrency, partial failure, staleness, auth edges, cross-feature, the
sequence half of UI state). That is the only territory nothing else in the pipeline reaches, and
it is where the budget belongs.
**Exit criterion:** the active class list, Band-C-weighted.
---
## Phase 3 — Route by engine
Each active class is tagged `[STATIC]` or `[SEQUENCE]`. Route accordingly:
- **Static — the default, always-on pass.** Don't re-derive these by hand. Run the bundled
scanner, which detects the `[STATIC]` code smells deterministically over the diff:
```bash
python scripts/scan_static.py # scans working-tree changes
python scripts/scan_static.py --base main # scans everything changed vs a base ref
python scripts/scan_static.py path/a.tsx # scans explicit files
```
It emits candidates tagged by catalog class and `confidence` (`high` → gate-grade,
`medium` = likely-real, `low` = a probably-missing-state hint), and marks whether each sits on a
changed line (`in_diff`). Covered today: float money, hand-rolled date math, boolean-soup /
bag-of-optionals state, permission-checked-once, swallowed catch/`except`, and missing
empty/error states. The scanner is a *candidate generator, not a verdict* — every candidate still
passes through the Phase 4 reachability gate, and `low` candidates in particular need the model
to confirm a state is genuinely missing and reachable. Read the printed snippet rather than
trusting the class label alone. Anything the scanner can't see statically (a smell it has no rule
for) you still inspect by hand — but the scanner is the floor, not the ceiling.
- **Execution — opt-in deep mode.** For `[SEQUENCE]` classes (interruption, concurrency, state
transitions) that can't be judged statically. Requires the substrate found in Phase 1 — **drive
the existing harness; never make the user build one.** Offer it; don't assume it.
Default to static-only unless the user asks for the deep pass or the change is high-blast-radius.
State which mode you're in.
**Exit criterion:** each finding-candidate assigned an engine.
---
## Phase 4 — Check against an oracle, then gate
For each candidate condition you need an **oracle** — how you know the result is *wrong* without a
hand-written spec. Use the cheapest that applies (full detail in the catalog's Oracle Layer):
1. **Crash/corruption floor** (free, near-zero false positives) — an unhandled exception, corrupted
or leaked state, or a contract violation in a plausible flow. No spec needed.
2. **Property oracles** — round-trip, idempotence, invariant, metamorphic.
3. **Differential** (opt-in) — run the condition against the prior version; a divergence the change
didn't intend is a regression.
Then the **reachability gate**, the precision valve: a condition becomes a *finding* only if you
can show the path is actually reachable for this feature. If you can't, drop it — an unreachable
"finding" is the exact false positive that gets the gate ignored. This applies to scanner
candidates too: a `high`-confidence smell on a code path nothing can reach is still a drop.
**Exit criterion:** every surviving finding is reachable and oracle-backed; the rest are dropped.
---
## Phase 5 — Verdict and minimized repro
Output a **verdict, not a report.** **PASS**, or **BLOCK** — and for each blocking finding, the
**shortest action sequence that reproduces it.** Minimize aggressively: a 3-step repro, not the
40-step path you found it on. "Something's wrong somewhere in these flows" is useless; "BLOCK:
submit → background app → reopen → submit → double charge" is the whole product.
Each minimized repro is exactly the input **Sleuth** wants — hand confirmed cases off to it for
diagnosis and fix. *Errata finds and blocks; Sleuth diagnoses.*
For each finding, report only: the **catalog class**, the **reachable repro**, and the **oracle**
that flags it. Nothing else — no padding, no speculation.
**Exit criterion:** PASS, or BLOCK with a minimized, reachable repro per finding.
---
## Worked examples
### A full BLOCK run
Feature under check: a "Claim daily reward" button in an Expo app — taps an endpoint that
increments a server-side balance, then shows the new total.
```
ERRATA STATUS
Mode: static (default)
Scope: ClaimRewardButton.tsx + claimReward() handler (diff of 2 files)
Active classes: B1 (repetition), C3/C7 (concurrent counting), C9b (states)
Findings: 2 reachable / 5 raw
Verdict: BLOCK — see repros
```
Phase 1 traits: writes state, performs an external call that can fail, increments a value that
matters, renders the result. Phase 2 activates B1, C7, B2, C9b (Band A skipped). Phase 3 static
scan returns five candidates:
```
[high] diff ClaimRewardButton.tsx:14 Swallowed error (B2-swallow)
[high] diff claimReward.ts:22 Float money (B6-money)
[medium] diff ClaimRewardButton.tsx:9 Permission checked once (C9e-permonce)
[low] diff ClaimRewardButton.tsx:31 Missing error state (C9b-error)
[low] ctx claimReward.ts:40 Missing empty state (C9b-empty)
```
Two survive Phase 4 as reachable, oracle-backed findings (the other three are handled below and in
the next example):
- **B1 — double-submit.** The button has no in-flight guard, so a fast double-tap fires
`claimReward()` twice before the first returns.
*Oracle:* idempotence — claiming once and claiming twice must leave the same balance; they don't.
*Repro:* `tap Claim → tap Claim again within ~300ms → balance increments twice`.
- **B2 — swallowed failure (scanner B2-swallow, line 14), promoted by reachability.** The `catch {}`
on the claim call means a network failure leaves the optimistic "+1" on screen while the server
never recorded it.
*Oracle:* crash/corruption floor — UI state and server state diverge with no signal.
*Repro:* `airplane mode → tap Claim → balance shows +1 → reopen app → balance unchanged`.
Verdict delivered:
```
BLOCK (2 findings)
1. [B1 repetition] double-claim
submit → submit (within ~300ms) → balance +2 from one reward
oracle: idempotence violated (claim×2 ≠ claim×1)
2. [B2 error path] optimistic update survives a failed write
airplane mode → tap Claim → UI shows +1 → reopen → server has +0
oracle: crash/corruption floor (UI vs server state diverge, silently)
Both repros are ready for Sleuth to diagnose.
```
Note the float-money `high` candidate (line 22) is real but the user owns severity — it's reported
plainly, not inflated into a blocker on its behalf.
### A killed finding (precision in action)
The scanner flagged `[medium] ClaimRewardButton.tsx:9 Permission checked once (C9e-permonce)` — a
camera-permission check in a mount-only effect. It looks like a Band-C foreground-revocation bug.
Reachability gate: the permission is requested at line 9, but nothing on this surface *uses* the
camera — the call is dead code left from a copy-pasted template, and the claim flow never reads the
permission result. No reachable path turns the stale permission into wrong behavior.
**Dropped.** Reported as nothing — not as a softened "you might want to look at the camera
permission." A finding that can't be tied to a reachable consequence is exactly the false positive
that trains the team to click through the gate. Silence here is the correct output. (If you think
the dead call is worth mentioning at all, it's a cleanup note, not an Errata finding — keep it out
of the verdict.)
---
## A note on cost
Errata is meant to be cheap enough that it never gets skipped. The static default needs no harness,
no setup, scopes to the diff, and leans on the bundled scanner for the mechanical checks — it
should feel like a fast lint pass you run on every feature. Execution mode is the deliberate,
opt-in deep dive for high-stakes changes. The structure costs nothing when the feature is simple;
it only does real work where the change actually has the surface for it. Don't pad a run to look
thorough — a clean PASS on a small diff is a complete, correct result.
~/.claude/skills/errata/references/forgotten-behaviors-catalog.md# The Forgotten-Behaviors Catalog
The set of mundane, foreseeable things real users (and real environments) do that developers
routinely fail to think through — and that therefore survive testing to surface in production.
This is the reference the skill checks a feature against.
## How to read this
Two axes run through the catalog:
- **Forgettability band (A / B / C)** — how reliably this class is *already* caught by linters,
types, and existing tests. This routes effort: don't spend budget where Band A already holds.
- **Applies-when trigger** — the feature trait that activates a class. This is what keeps the
skill cheap: a read-only screen never gets checked for double-submit.
The classification borrows the key distinction from Orthogonal Defect Classification: a defect's
*type* (what's wrong in the code) is separate from its *trigger* (the condition that surfaces it).
This catalog is a **trigger taxonomy** — it enumerates the conditions, because those are what a
dev fails to imagine, and what existing tests fail to exercise.
The skill's loop: detect feature traits → activate triggered classes → generate the condition →
check against an oracle (see the Oracle Layer at the end) → report only reachable, real failures.
## Contents (routing table — open only what the traits activate)
This TOC *is* the scoping tool. Match the detected traits to the *applies-when* column and read
only those entries; do not load the whole catalog. `[S]` = STATIC engine (covered by
`scan_static.py`), `[Q]` = SEQUENCE engine (needs execution), `[C]` = a completeness gap.
**Band A — table stakes (skip unless something cheap flags it)**
- `A1` Empty / zero / nothing — *reads or displays any value/collection* `[S]`
- `A2` Basic input shape — *accepts typed or pasted input* `[S]`
- `A3` Boundary / off-by-one — *has any limit, range, or pagination* `[S]`
**Band B — frequently forgotten**
- `B1` Repetition / idempotency — *performs a write, mutation, or external call* `[Q]`
- `B2` Error & failure paths — *calls anything that can fail* `[S]` (swallow) `[Q]` (partial)
- `B3` Lifecycle / cleanup — *acquires a resource, subscription, listener, or lock* `[S/Q]`
- `B4` Volume / scale — *renders/processes collections that grow over time* `[Q]`
- `B5` Config / deployment / environment — *reads config, flags, locale, secrets* `[S]`
- `B6` Real-world data assumptions — *stores/validates/formats/sorts real-world data* `[S]`
(money, time, names, email, addresses, phone, geo, i18n, networks, paths) — pair with corpora
**Band C — almost always forgotten (the prize; weight here)**
- `C1` Interruption / partial completion — *any multi-step flow or in-flight operation* `[Q]`
- `C2` Ordering / sequence — *multiple actions touching shared state* `[Q]`
- `C3` Concurrency (second-actor) — *same resource touched by 2 sessions/devices/users* `[Q]`
- `C4` Distributed / partial failure — *spans services, a queue, any network boundary* `[Q]`
- `C5` State staleness / cache / sync — *caches, mirrors, or derives from a source of truth* `[Q]`
- `C6` Identity / auth / permission edges — *access-controlled or resource-scoped* `[S/Q]`
- `C7` Counting / money / aggregation under concurrency — *counts/sums/decrements* `[Q]`
- `C8` Cross-feature interaction — *shares state/resources/side effects with another feature* `[Q]`
- `C9` UI state completeness — *any data-driven view*; splits by engine:
- `C9a` State stack (ideal/loading/partial/error/empty) `[C]`
- `C9b` Empty/loading/error are not single states `[C]`
- `C9c` Impossible-state code smells (boolean soup, optional bag, perm-once) `[S]`
- `C9d` Combined & transition states `[Q]`
- `C9e` Mobile / React Native lifecycle (permission richness, foreground, cold start) `[S/Q]`
- `C9f` Content & environment variations (type scaling, RTL, dark mode, overflow) `[S/C]`
**Then, regardless of class:** `Oracle Layer` (how you decide a triggered behavior is a bug) and
`Concrete test corpora` (don't hand-roll B6 inputs).
---
## Band A — Table stakes (usually already caught; low marginal value)
Included for completeness so the skill can certify it considered them. Assume linters, type
systems, and existing unit tests cover most of this.
### A1. Empty / zero / nothing
*Applies when:* the feature reads or displays any value or collection.
- Zero items; exactly one item (singular/plural copy; deleting the last one — empty again).
- Zero as a *value* vs null/undefined — the falsy trap where `if (count)` silently skips zero.
- Empty string vs null vs whitespace-only.
- First run / cold start: no data, no history, brand-new account.
- "Had data, now all deleted" — distinct from never-had-data.
### A2. Basic input shape
*Applies when:* the feature accepts typed or pasted input.
- Leading/trailing whitespace; smart quotes; control characters from a paste.
- Negative, zero, or decimal where a positive integer was assumed.
- Numbers-as-strings and type-coercion surprises.
- Characters that break parsing or escaping (`"`, `<`, `&`, null byte).
*(The deeper data-assumption failures live in Band B — "Real-world data".)*
### A3. Boundary / off-by-one
*Applies when:* the feature has any limit, range, or pagination.
- Exactly at the limit; one over; one under.
- Min/max of a range; the limit changing while the user sits at the boundary.
---
## Band B — Frequently forgotten (solid value)
These require imagining the unhappy path, the heavy user, or messy real-world data. Skipped
under time pressure rather than out of ignorance.
### B1. Repetition / idempotency
*Applies when:* the feature performs a write, mutation, or external call.
- Double-click / double-submit before the first call returns.
- Retry after a *perceived* failure that actually succeeded server-side.
- Refresh during an in-flight operation.
- The same request arriving twice (network retry, back-then-forward).
- Creating a duplicate of something meant to be unique.
### B2. Error & failure paths
*Applies when:* the feature calls anything that can fail (network, DB, third party, disk).
- Dependency down, slow, or returning malformed data.
- Partial success (3 of 5 saved) — what state is the user left in?
- The error handler itself throws.
- Silent swallow: a `catch` that hides the failure and continues as if fine.
- Failure *after* a side effect already happened (charged but not recorded).
- Retry storm: failure → retry → more load → more failure (the cascading-failure seed).
### B3. Lifecycle / cleanup
*Applies when:* the feature acquires a resource, subscription, listener, or lock.
- Resource not released on the *error* path (handle, lock, connection, timer).
- Listener/subscription leak across remounts or repeated entries.
- Async resolves *after* the component/context is gone (the "setState after unmount" classic).
### B4. Volume / scale
*Applies when:* the feature renders or processes collections that grow over time.
- The five-year account vs the demo account; the 10,000-item list.
- A very long single string overflowing layout.
- Work that's fine on the dev's three records and quadratic at production size.
- Many rapid actions in succession.
### B5. Configuration / deployment / environment
*Applies when:* the feature reads config, flags, locale, secrets, or env-specific values.
- A feature flag in an unexpected combination with another flag.
- Config consumed without validation (a leading cause of real deploy-time outages).
- A default that's wrong for a real user (locale, timezone, currency, units).
- Works in dev, breaks in prod (config, secrets, real latency, real scale).
- Missing optional config silently treated as present.
### B6. Real-world data assumptions (the "falsehoods" canon)
*Applies when:* the feature stores, validates, formats, sorts, or compares any real-world data.
This is the largest forgotten branch — each item below is a *false assumption* devs encode by
default. Pair this section with concrete corpora (see References) rather than hand-rolled cases.
- **Time & dates.** Days aren't always 24h (DST); a day/week/month can begin and end in
different years; a wall-clock time can occur twice or never (DST transitions); "store
everything in UTC" does not solve future-event scheduling; leap seconds and leap days exist;
the server's timezone is not the user's; timezone *rules themselves* change. Treat any
hand-rolled date logic as a finding by default.
- **Names & identity.** Names aren't ASCII, aren't first/last, can change, can be a single
character or very long, can contain spaces/punctuation, and may not be unique. Gender and
family structure resist enum modeling. (The literal user named "Null" breaks naive systems.)
- **Email.** "Exactly one `@`, validate by regex" is wrong — the RFC permits far more than
intuition allows. Don't validate by regex; send a confirmation.
- **Addresses.** No postal codes in some countries; cities without streets; non-Latin scripts;
addresses that are landmarks, not lines. Regex and addresses don't mix.
- **Phone numbers.** Length, format, and country rules vary wildly; a number isn't a stable
unique key. Use a real library, not a pattern.
- **Geography & coordinates.** Coordinate systems differ; the date line and poles are edge
cases; place names are non-unique and non-Latin; borders and country counts change.
- **Internationalization / Unicode.** String length ≠ grapheme count (combining chars,
surrogate pairs, emoji); case-folding is locale-dependent; normalization forms differ;
substring/truncation can split a grapheme; RTL and bidi reorder display.
- **Money & numbers.** Never use floats for money (`0.1 + 0.2 ≠ 0.3`); use decimals or integer
minor units; mind rounding direction and currency-specific minor-unit counts. Separating
"pennies from dollars" has caused real 100× and $25k errors.
- **Networks / IP.** Multiple valid notations for an IP; IPv6 exists; the Fallacies of
Distributed Computing (the network is *not* reliable, latency *not* zero, bandwidth *not*
infinite, topology *not* static) are all assumptions devs silently bake in.
- **File paths, URLs, CSVs, versions, pagination.** Each has its own canon of broken
assumptions (Windows vs POSIX paths; URL component parsing; CSV quoting/embedded newlines;
version-string ordering; pagination under concurrent insertion/deletion).
---
## Band C — Almost always forgotten (the prize)
Every class here requires imagining one of three things a dev writing a function in isolation
never does: **a sequence of actions**, **a second actor**, or **time passing between steps**.
This is the production-only family, and it is exactly the surface that unit tests and code review
structurally cannot reach. Spend the skill's budget here.
### C1. Interruption / partial completion
*Applies when:* the feature has any multi-step flow or in-flight operation.
- App backgrounded mid-transaction (mobile); OS kills it before resume.
- Network drops mid-operation — what is the recoverable state?
- User navigates away / closes the tab mid-flow.
- Browser back button mid-wizard with state half-applied.
- Session/token expires mid-flow.
- Client-side timeout fires but the operation completed server-side.
### C2. Ordering / sequence (the dev assumed an order)
*Applies when:* the feature has multiple actions touching shared state.
- Actions taken out of the intended order (step 3 before step 1).
- Acting on a view that is now stale (optimistic UI lying about server truth).
- Editing something deleted in another tab or by another path.
- Undo after a dependent action already consumed the thing.
- Two operations racing — which finishes first, and does the code assume?
### C3. Concurrency (the second-actor sub-taxonomy)
*Applies when:* the same resource can be touched by two sessions, devices, users, or requests.
These look correct in isolation and pass most tests; they stay hidden until concurrency rises,
so a feature fine for a handful of users can fail catastrophically as it succeeds and scales.
- **Lost update (read-modify-write).** Two actors read, compute, write back; one silently
overwrites the other. The canonical counter/balance bug.
- **Check-then-act / TOCTOU.** State observed, then acted on assuming it's unchanged — the gap
between check and use is the bug (and a classic security hole: cancel-but-still-paid,
permission-checked-then-request-modified, reserve-then-oversell).
- **Last-write-wins clobber.** Two users edit the same record; one's edit vanishes with no
conflict surfaced.
- **Same user, two contexts.** Two tabs / two devices with diverging local state.
- **Deadlock / livelock.** Circular waits, or threads endlessly yielding without progress.
- **Order/atomicity violation.** A multi-step operation interrupted partway leaves an invalid
intermediate state visible.
### C4. Distributed / partial failure
*Applies when:* the feature spans services, a queue, or any network boundary.
- Partial failure: some calls in a fan-out succeed, some fail — is the whole left consistent?
- Non-idempotent retries double-apply an effect (the retry safety property).
- Cascading failure: one slow dependency exhausts a pool and takes down callers.
- Event-driven assumptions: messages arrive out of order, more than once, or never; "exactly
once" is usually a fiction.
### C5. State staleness / cache / sync
*Applies when:* the feature caches, mirrors, or derives data from a source of truth.
- Cached data shown after the underlying thing changed.
- Optimistic update the server then rejects — does the UI roll back cleanly?
- Two sources of truth quietly disagreeing.
- A permission or flag flip not reflected until reload.
### C6. Identity / auth / permission edges
*Applies when:* the feature is access-controlled or resource-scoped.
- Logged-out user hitting a logged-in path via deep link or bookmark.
- Permissions changed mid-session; the open page still assumes the old ones.
- Accessing another user's resource by changing an ID in the URL (IDOR).
- Account in a weird state: unverified, suspended, trial-expired, mid-migration.
### C7. Counting / money / aggregation under concurrency
*Applies when:* the feature counts, sums, or decrements anything that matters.
- Concurrent decrements overselling inventory / double-spending a credit.
- Aggregate computed over a stale or partial set.
- Negative balance or quantity reached by an unguarded concurrent path.
- Sums that don't reconcile (rounding, silently excluded items).
### C8. Cross-feature interaction
*Applies when:* the feature shares state, resources, or side effects with another feature.
- This feature's state colliding with another's over a shared resource.
- Notifications/emails fired by an action the user then undid.
- A data migration this feature didn't know about.
- A side effect here that another feature's invariant silently depends on.
### C9. UI state completeness (frontend) — expanded section
The richest forgotten zone for any UI surface, and it splits cleanly along the static-vs-execute
line: some sub-classes are **[STATIC]** code smells detectable without running anything, others
are **[SEQUENCE]** bugs that need execution, and others are **[COMPLETENESS]** gaps (a state the
designer never drew). Each item below is tagged so the skill routes it to the right engine.
Why weight this class heavily, especially for AI-written code: forgotten states are
underrepresented in training data, so AI-generated frontends skip them at unusually high rates —
one 2025 analysis of AI-built dashboards found near-total absence of empty and error states and a
universally lazy loading state, attributed to training data being dominated by happy-path
tutorials rather than production edge handling.
#### C9a. State completeness — the UI Stack [COMPLETENESS]
*Applies when:* the feature is any data-driven view.
The canonical five states are **ideal, loading, partial, error, empty** — a UI missing any of
them feels broken. The two most-skipped:
- **Partial:** sparse/incomplete data (one item where hundreds were assumed; some fields present,
others missing) so features like pagination never activate and layout collapses.
- Each of the five fragments further (below), and the fragments are where bugs actually live.
#### C9b. Empty / loading / error are not single states [COMPLETENESS]
*Applies when:* the view fetches or mutates data.
- **Empty is ≥4 distinct states**, each with different copy and correct action: first-run /
onboarding (never had data), no-search-results (query matched nothing), user-cleared (deleted
everything), no-permission, and the bug-prone *error-rendered-as-empty* (failed load shown as
"nothing here").
- **Loading is several:** initial load (skeleton), refetch-with-stale-data-visible, load-more /
pagination, background refresh. Sub-bugs: flash-of-spinner on a sub-100ms load; spinner with no
timeout; discarding good data to show a spinner on refetch.
- **Error is several:** page-level vs section-level vs field-level; retryable vs fatal; validation
vs system; partial-batch failure; error-with-last-valid-data (keep showing the last good value).
#### C9c. Impossible-state code smells [STATIC]
*Applies when:* component/store models one async resource. **Detectable without execution** — this
is the cause of "spinner AND error at once" rendering bugs, and a high-precision static finding.
- **Boolean soup:** multiple correlated flags (`isLoading`, `isError`, `isSuccess`) modeling one
resource create 2^n states, most invalid; nothing prevents `isLoading && isError`.
- **Bag of optionals:** `{ loading?, data?, error? }` permits success-without-data and
loading-carrying-an-error.
- **Permission/state checked once at mount** and trusted thereafter (see C9e mobile).
- Healthy form (the suggested fix, not a bug): a discriminated union / state machine where
invalid combinations don't compile — and which can model the *legitimate* combined states
explicitly (`loading` with `staleData`, `success` with `isStale`, `error` with `lastValidData`).
#### C9d. Combined & transition states [SEQUENCE]
*Applies when:* the view moves between states over time.
- Combined: loading-while-showing-stale, success-but-stale, error-but-have-cached-data.
- Transitions: ideal → empty (deleted the last item), success → error (a refetch fails after data
was shown), rapid back-and-forth between states. These need execution to surface.
#### C9e. Mobile / React Native lifecycle states [SEQUENCE / STATIC mix]
*Applies when:* the surface runs in a native app shell (e.g. Expo/RN).
- **Permission richness [STATIC for "checked once", SEQUENCE for revocation]:** the real status
set is granted / denied / **blocked** (permanently denied — can't re-prompt, must route to
Settings) / **unavailable** (not on device) / restricted / undetermined. A one-time grant is
auto-revoked within ~30–60s of the app being terminated or backgrounded-and-unused, and users
can toggle permissions in Settings while backgrounded — so a permission checked only at mount
and trusted afterward is a latent bug; re-check on foreground via AppState.
- App backgrounded/foregrounded mid-flow (auth or data changed while away).
- Deep-link or push-notification **cold start** landing directly on a deep screen with no nav
stack and no warm session.
- Keyboard covering the input or submit button; orientation change mid-flow; safe-area/notch.
- Offline → online transitions; airplane mode mid-action; slow network.
- OTA-update / app-version skew mid-session.
#### C9f. Content & environment variations [COMPLETENESS / STATIC]
*Applies when:* the surface renders text, images, or themed UI.
- Dynamic type / font scaling; RTL layout; dark mode or a system-theme flip mid-session.
- Very long content and truncation; broken/missing images; localized text length overflowing
layout (e.g. German); zero/one/many pluralization copy.
---
## The Oracle Layer — deciding a triggered behavior is actually a bug
Generating a forgotten condition is half the job; the other half is knowing the result is wrong
*without* a hand-written spec. Use these oracles, cheapest first:
1. **Crash / corruption floor (free, near-zero false positives).** Did it throw an unhandled
exception or rejection, corrupt state, leak state between sequences, or violate its own type
contract? No spec needed — an exception in a plausible user flow is essentially never
intended. This catches most of Band C with no user input.
2. **Property oracles (partial oracles — no full spec required).** A property defines a
relationship that must hold, sidestepping the "what's the exact correct output?" problem:
- **Round-trip:** `decode(encode(x)) == x` (serialization, parse/format, save/load).
- **Idempotence:** doing it twice equals doing it once (retries, normalization, dedupe,
`set` operations) — directly validates B1 and C4.
- **Invariant:** something preserved across an operation (collection size after a map, total
after a transfer, balance never negative) — directly validates C7.
- **Metamorphic:** a known input change implies a known output change, when you can't compute
the right answer directly (reorder input → same sorted output; sub-path of a shortest path
is itself shortest). Powerful for logic with no easy reference answer.
3. **Differential oracle (opt-in, higher cost).** Run the condition against the prior version;
any divergence the change wasn't meant to cause is a regression. The old version is the
oracle for everything the feature didn't intend to alter. Note the known weakness: an
independent *reimplementation* tends to share faults on the same inputs, so prefer
prior-version diffing or a deliberately-simple reference, not a parallel rewrite.
Mapping: Band A/B input classes → round-trip + invariant. Idempotency (B1) → idempotence.
Counting/money (C7) → invariant. Anything without a clean property → crash floor + differential.
---
## Concrete test corpora (don't hand-roll these)
For the data classes in B6, feed real corpora rather than inventing cases:
- **Big List of Naughty Strings** — inputs with a high probability of breaking input handling.
- **i18n / international name test data** — real diverse names and scripts.
- **libphonenumber / libaddressinput** — reference validation for phones and addresses.
- The "falsehoods programmers believe" articles per domain — use as the assumption checklist
behind each B6 bullet.
---
## Notes for turning this into the skill
- **Triggers do the scoping.** Feature-trait detection up front (reads? writes? holds state
across actions? multi-user? calls a dependency? renders data?) maps traits → active classes.
This is what keeps it cheap and stops irrelevant checks.
- **Band C is the differentiator.** Bands A–B overlap with linters, types, and existing tests.
A version shipping only Band C + the oracle layer would still beat the entire generic-review
tier, because nothing else explores sequence / second-actor / time.
- **Every check needs a reachability gate.** A class fires as a finding only if the path is
actually reachable for this feature — otherwise you reintroduce the false-positive fatigue
that gets gates ignored.
- **Route by engine, not just by class.** Many classes are mixed: tag each check `[STATIC]`
(a code smell detectable without running — e.g. boolean-soup state, permission-checked-once,
float money, hand-rolled date math) vs `[SEQUENCE]` (needs execution — interruption,
concurrency, state transitions). The static-tagged checks are the cheap always-on default;
the sequence-tagged ones are the opt-in execution mode. C9 is the worked example of this split.
- **The AI-generated-code angle is a wedge.** Forgotten behaviors are, by definition,
underrepresented in training data, so AI-written features skip them at unusually high rates.
A gate aimed at exactly this gap is most valuable precisely where code is AI-authored.
- **Version the catalog.** Every real production escape becomes a new entry or a sharpened
trigger. This is the asset that compounds — and the same loop ODC was built around: turn the
defect stream into a measurement that improves the process.
~/.claude/skills/errata/scripts/scan_static.py#!/usr/bin/env python3
"""
scan_static.py — Errata's static engine.
Runs the deterministic, [STATIC]-tagged checks from the forgotten-behaviors
catalog against a changed surface, so the model does not re-derive them by hand
on every run. Pure stdlib: nothing to install, nothing to skip.
Design ethos matches the skill: PRECISION OVER COMPLETENESS. Every detector is
tuned to rarely false-positive. Findings carry a confidence:
high — gate-grade; a true positive almost every time it fires.
medium — very likely real, but worth a glance.
low — a hint a state is probably missing; the model must confirm
reachability before it ever becomes a finding.
The script never decides severity and never emits a verdict. It produces
*candidates* tagged by catalog class. Phase 4's reachability gate still runs.
Usage:
python scripts/scan_static.py # scan working-tree changes (git diff)
python scripts/scan_static.py --base main # scan everything changed vs `main`
python scripts/scan_static.py src/a.tsx b.ts # scan explicit files
python scripts/scan_static.py --json # JSON only (default prints both)
Exit code is always 0 — this is a candidate generator, not a CI gate. The
verdict belongs to the model and the user.
"""
from __future__ import annotations
import argparse
import json
import os
import re
import subprocess
import sys
from dataclasses import dataclass, asdict, field
from typing import Iterable
JS_EXT = {".js", ".jsx", ".ts", ".tsx", ".mjs", ".cjs"}
PY_EXT = {".py"}
SCANNABLE = JS_EXT | PY_EXT
MAX_BYTES = 2_000_000 # skip generated/minified monsters; they aren't the diff
@dataclass
class Finding:
catalog_id: str
klass: str
file: str
line: int
snippet: str
confidence: str # high | medium | low
why: str
in_diff: bool = False
engine: str = "STATIC"
def to_dict(self) -> dict:
d = asdict(self)
d["class"] = d.pop("klass")
return d
# --------------------------------------------------------------------------- #
# git plumbing — scope to the change, and learn which lines are actually new. #
# --------------------------------------------------------------------------- #
def _git(args: list[str]) -> str | None:
try:
out = subprocess.run(
["git", *args],
capture_output=True, text=True, check=True,
)
return out.stdout
except (subprocess.CalledProcessError, FileNotFoundError):
return None
def changed_files(base: str | None) -> list[str]:
spec = [f"{base}...HEAD"] if base else []
out = _git(["diff", "--name-only", *spec, "--diff-filter=d"]) # d = exclude deletes
if out is None:
return []
files = [f.strip() for f in out.splitlines() if f.strip()]
if not files and base is None:
# nothing staged/unstaged differs from HEAD; fall back to last commit
out = _git(["diff", "--name-only", "HEAD~1...HEAD", "--diff-filter=d"])
files = [f.strip() for f in out.splitlines() if f.strip()] if out else []
return files
def added_line_ranges(base: str | None) -> dict[str, list[tuple[int, int]]]:
"""Map file -> list of (start, end) line ranges that the diff adds."""
spec = [f"{base}...HEAD"] if base else []
out = _git(["diff", "-U0", *spec])
ranges: dict[str, list[tuple[int, int]]] = {}
if out is None:
return ranges
cur: str | None = None
hunk = re.compile(r"^@@ -\d+(?:,\d+)? \+(\d+)(?:,(\d+))? @@")
for line in out.splitlines():
if line.startswith("+++ b/"):
cur = line[6:].strip()
ranges.setdefault(cur, [])
elif line.startswith("@@") and cur is not None:
m = hunk.match(line)
if m:
start = int(m.group(1))
count = int(m.group(2)) if m.group(2) else 1
if count > 0:
ranges[cur].append((start, start + count - 1))
return ranges
def in_ranges(line_no: int, ranges: list[tuple[int, int]]) -> bool:
return any(a <= line_no <= b for a, b in ranges)
# --------------------------------------------------------------------------- #
# helpers #
# --------------------------------------------------------------------------- #
def line_of(text: str, idx: int) -> int:
return text.count("\n", 0, idx) + 1
def snippet_at(lines: list[str], line_no: int) -> str:
return lines[line_no - 1].strip()[:160] if 0 < line_no <= len(lines) else ""
def looks_rn(text: str) -> bool:
return bool(re.search(r"from\s+['\"](react-native|expo[\w/-]*)['\"]", text))
MONEY = re.compile(
r"\b\w*(price|amount|total|subtotal|cost|balance|fee|tax|vat|discount|"
r"salary|wage|payroll|payment|charge|refund|credit|debit|cents|dollars|"
r"usd|eur|gbp|money|currenc|invoice|payout|deposit)\w*\b",
re.IGNORECASE,
)
DECIMAL = re.compile(r"\b\d+\.\d+\b")
ARITH = re.compile(r"[-+*/]")
# --------------------------------------------------------------------------- #
# detectors #
# --------------------------------------------------------------------------- #
def detect_float_money(path: str, text: str, lines: list[str]) -> Iterable[Finding]:
for i, ln in enumerate(lines, 1):
code = ln.split("//", 1)[0]
if not MONEY.search(code):
continue
# parseFloat / Number on something money-shaped: high.
if re.search(r"parseFloat\s*\(", code) or re.search(r"\bNumber\s*\(", code):
yield Finding(
"B6-money", "Float money", path, i, ln.strip()[:160], "high",
"parseFloat/Number used on a money-named value — floats can't "
"represent currency exactly (0.1 + 0.2 != 0.3). Use integer "
"minor units or a decimal type.",
)
continue
# money name + decimal literal + arithmetic on the same line: high.
if DECIMAL.search(code) and ARITH.search(code):
yield Finding(
"B6-money", "Float money", path, i, ln.strip()[:160], "high",
"Arithmetic on a money-named value with a float literal. Money "
"in floats accumulates rounding error; use minor units.",
)
continue
# cents<->dollars toggling by *100 / /100: medium (the $25k-error smell).
if re.search(r"[*/]\s*100\b", code):
yield Finding(
"B6-money", "Float money", path, i, ln.strip()[:160], "medium",
"Manual *100 / /100 on a money value — the classic "
"pennies<->dollars conversion that has caused real 100x errors. "
"Confirm a single, tested money type owns this.",
)
# toFixed used to 'round' money: medium.
elif re.search(r"\.toFixed\s*\(\s*2\s*\)", code):
yield Finding(
"B6-money", "Float money", path, i, ln.strip()[:160], "medium",
"toFixed(2) formats but does not round money correctly and "
"still operates on a float. Round in minor units before display.",
)
DATE_MS = re.compile(
r"(24\s*\*\s*60\s*\*\s*60\s*\*\s*1000" # ms per day, common orderings
r"|60\s*\*\s*60\s*\*\s*24\s*\*\s*1000"
r"|1000\s*\*\s*60\s*\*\s*60\s*\*\s*24"
r"|\b86_?400_?000\b" # ms per day literal
r"|\b86_?400\b\s*\*\s*1000)"
)
DATE_COMPONENT_MATH = re.compile(
r"\.(setDate|setMonth|setFullYear|setHours)\s*\(\s*[^)]*\.(getDate|getMonth|"
r"getFullYear|getHours)\s*\(\)\s*[-+]"
)
DATE_EPOCH_MATH = re.compile(r"(getTime\s*\(\)|Date\.now\s*\(\))\s*[-+]")
def detect_handrolled_date(path: str, text: str, lines: list[str]) -> Iterable[Finding]:
for i, ln in enumerate(lines, 1):
code = ln.split("//", 1)[0]
if DATE_MS.search(code):
yield Finding(
"B6-time", "Hand-rolled date math", path, i, ln.strip()[:160], "high",
"Manual milliseconds-per-day arithmetic. Days aren't always 24h "
"(DST), so adding 86,400,000ms skips/repeats an hour twice a year. "
"Use a date library's day arithmetic.",
)
elif DATE_COMPONENT_MATH.search(code):
yield Finding(
"B6-time", "Hand-rolled date math", path, i, ln.strip()[:160], "high",
"Hand-rolled date-component arithmetic. Month/day rollover and "
"DST make this wrong at boundaries; use a library.",
)
elif DATE_EPOCH_MATH.search(code) and re.search(r"day|hour|week|month|expire|ttl|age", code, re.I):
yield Finding(
"B6-time", "Hand-rolled date math", path, i, ln.strip()[:160], "medium",
"Epoch arithmetic used to derive a duration/expiry. Verify it "
"isn't assuming fixed-length days.",
)
# correlated async-resource flags — boolean soup (C9c)
USESTATE = re.compile(r"\b(?:const|let)\s*\[\s*(\w+)\s*,", re.IGNORECASE)
USESTATE_CALL = re.compile(r"=\s*useState\b")
LOADING = re.compile(r"^(is)?(loading|fetching|pending|busy)$", re.I)
ERRORF = re.compile(r"^(is)?error$|^err$|^hasError$", re.I)
SUCCESS = re.compile(r"^(is)?(success|succeeded|loaded|done|ready)$", re.I)
def detect_boolean_soup(path: str, text: str, lines: list[str]) -> Iterable[Finding]:
if os.path.splitext(path)[1] not in JS_EXT:
return
# Collect useState boolean-ish flags with their line numbers.
flags: list[tuple[int, str, str]] = [] # (line, name, concept)
for i, ln in enumerate(lines, 1):
if not USESTATE_CALL.search(ln):
continue
m = USESTATE.search(ln)
if not m:
continue
name = m.group(1)
if LOADING.match(name):
flags.append((i, name, "loading"))
elif ERRORF.match(name):
flags.append((i, name, "error"))
elif SUCCESS.match(name):
flags.append((i, name, "success"))
# Cluster flags within ~40 lines (a rough same-component window).
flags.sort()
n = len(flags)
for a in range(n):
window = [flags[a]]
for b in range(a + 1, n):
if flags[b][0] - flags[a][0] <= 40:
window.append(flags[b])
concepts = {c for _, _, c in window}
if len(concepts) >= 2:
names = ", ".join(sorted({nm for _, nm, _ in window}))
yield Finding(
"C9c-boolsoup", "Impossible-state boolean soup", path, flags[a][0],
snippet_at(lines, flags[a][0]), "high",
f"Separate useState flags ({names}) model one async resource. "
"2^n combinations exist, most invalid — nothing prevents "
"isLoading && isError rendering at once. Model it as one "
"discriminated union / state machine.",
)
break # one finding per component cluster is enough
OPTIONAL_BAG = re.compile(
r"\{\s*(?=[^}]*\bdata\s*\?)(?=[^}]*\b(loading|isLoading)\s*\?)"
r"(?=[^}]*\b(error|isError)\s*\?)[^}]*\}"
)
def detect_optional_bag(path: str, text: str, lines: list[str]) -> Iterable[Finding]:
if os.path.splitext(path)[1] not in JS_EXT:
return
for m in OPTIONAL_BAG.finditer(text):
i = line_of(text, m.start())
yield Finding(
"C9c-optbag", "Bag of optionals", path, i, snippet_at(lines, i), "medium",
"A { data?, loading?, error? } shape permits success-without-data "
"and loading-carrying-an-error. Prefer a discriminated union.",
)
PERM_API = re.compile(
r"(request\w*Permission|get\w*Permission|check\w*Permission|hasPermission|"
r"Permissions\.\w+|requestPermissionsAsync|getPermissionsAsync)",
re.IGNORECASE,
)
MOUNT_EFFECT = re.compile(r"useEffect\s*\([^,]*,\s*\[\s*\]\s*\)", re.DOTALL)
def detect_permission_once(path: str, text: str, lines: list[str]) -> Iterable[Finding]:
if os.path.splitext(path)[1] not in JS_EXT or not looks_rn(text):
return
if not PERM_API.search(text):
return
if "AppState" in text:
return # likely re-checks on foreground — don't cry wolf
if not MOUNT_EFFECT.search(text):
return
m = PERM_API.search(text)
i = line_of(text, m.start())
yield Finding(
"C9e-permonce", "Permission checked once", path, i, snippet_at(lines, i),
"medium",
"Permission is checked in a mount-only effect with no AppState listener. "
"A one-time grant auto-revokes ~30-60s after backgrounding, and users "
"can flip it in Settings while away. Re-check on foreground.",
)
EMPTY_CATCH_JS = re.compile(r"catch\s*(\([^)]*\))?\s*\{\s*\}")
EMPTY_CATCH_ARROW = re.compile(r"\.catch\s*\(\s*\(?[^)]*\)?\s*=>\s*\{\s*\}\s*\)")
LOGONLY_CATCH = re.compile(
r"catch\s*(\([^)]*\))?\s*\{\s*console\.\w+\([^;]*\);?\s*\}"
)
def detect_swallowed_catch_js(path: str, text: str, lines: list[str]) -> Iterable[Finding]:
for m in EMPTY_CATCH_JS.finditer(text):
i = line_of(text, m.start())
yield Finding(
"B2-swallow", "Swallowed error", path, i, snippet_at(lines, i), "high",
"Empty catch block hides the failure and continues as if fine. The "
"user is left in an undefined state with no signal.",
)
for m in EMPTY_CATCH_ARROW.finditer(text):
i = line_of(text, m.start())
yield Finding(
"B2-swallow", "Swallowed error", path, i, snippet_at(lines, i), "high",
"Empty .catch() swallows a rejected promise — the failure path is "
"silently a no-op.",
)
for m in LOGONLY_CATCH.finditer(text):
i = line_of(text, m.start())
yield Finding(
"B2-swallow", "Swallowed error", path, i, snippet_at(lines, i), "medium",
"catch only logs and continues — no recovery, no user-visible error, "
"no rethrow. Decide the real failure state.",
)
BARE_EXCEPT = re.compile(r"except\s*:\s*\n\s*pass", re.MULTILINE)
TYPED_EXCEPT_PASS = re.compile(r"except\s+[\w.]+(?:\s+as\s+\w+)?\s*:\s*\n\s*pass", re.MULTILINE)
def detect_swallowed_except_py(path: str, text: str, lines: list[str]) -> Iterable[Finding]:
if os.path.splitext(path)[1] not in PY_EXT:
return
for rx, conf in ((BARE_EXCEPT, "high"), (TYPED_EXCEPT_PASS, "high")):
for m in rx.finditer(text):
i = line_of(text, m.start())
yield Finding(
"B2-swallow", "Swallowed error", path, i, snippet_at(lines, i), conf,
"except: pass swallows the failure and continues. At minimum log "
"and decide the recovery state.",
)
MAP_RENDER = re.compile(r"\.map\s*\(")
FETCHES = re.compile(r"(useQuery|useSWR|fetch\s*\(|axios|useEffect)", re.I)
# Structural signals only — never prose/comments, which produce both false
# positives ("// handle the empty case later") and false negatives.
HAS_EMPTY = re.compile(r"(\.length\s*===?\s*0|\.length\s*<\s*1|\bisEmpty\b|<Empty|EmptyState)")
HAS_ERROR_UI = re.compile(r"(\bisError\b|\bhasError\b|<Error|ErrorBoundary|ErrorState)")
def _strip_comments(text: str) -> str:
# Replace comments with blanks but keep newline count intact, so line
# numbers computed against the stripped text still match the real file.
text = re.sub(r"/\*.*?\*/", lambda m: "\n" * m.group(0).count("\n"),
text, flags=re.DOTALL)
text = re.sub(r"//[^\n]*", "", text)
return text
def detect_missing_states(path: str, text: str, lines: list[str]) -> Iterable[Finding]:
if os.path.splitext(path)[1] not in JS_EXT:
return
code = _strip_comments(text)
if not (MAP_RENDER.search(code) and FETCHES.search(code)):
return
text = code # check empty/error signals against comment-free source
m = MAP_RENDER.search(text)
i = line_of(text, m.start())
if not HAS_EMPTY.search(text):
yield Finding(
"C9b-empty", "Missing empty state", path, i, snippet_at(lines, i), "low",
"Renders fetched data via .map but no empty-state branch is visible. "
"Empty is >=4 distinct states (first-run, no-results, user-cleared, "
"error-as-empty). Confirm one is handled and reachable.",
)
if not HAS_ERROR_UI.search(text):
yield Finding(
"C9b-error", "Missing error state", path, i, snippet_at(lines, i), "low",
"Fetches and renders but no error branch is visible. A failed load "
"likely renders as a permanent empty/blank screen.",
)
DETECTORS = [
detect_float_money,
detect_handrolled_date,
detect_boolean_soup,
detect_optional_bag,
detect_permission_once,
detect_swallowed_catch_js,
detect_swallowed_except_py,
detect_missing_states,
]
# --------------------------------------------------------------------------- #
# driver #
# --------------------------------------------------------------------------- #
def scan_file(path: str) -> list[Finding]:
try:
if os.path.getsize(path) > MAX_BYTES:
return []
with open(path, "r", encoding="utf-8", errors="replace") as fh:
text = fh.read()
except (OSError, ValueError):
return []
lines = text.splitlines()
out: list[Finding] = []
for det in DETECTORS:
try:
out.extend(det(path, text, lines))
except re.error:
continue
return out
def resolve_targets(args) -> tuple[list[str], str]:
if args.paths:
return [p for p in args.paths if os.path.splitext(p)[1] in SCANNABLE], "paths"
files = changed_files(args.base)
files = [f for f in files if os.path.splitext(f)[1] in SCANNABLE and os.path.exists(f)]
return files, ("git-diff" if not args.base else f"git-diff:{args.base}")
def main() -> int:
ap = argparse.ArgumentParser(description="Errata static engine.")
ap.add_argument("paths", nargs="*", help="explicit files to scan")
ap.add_argument("--base", help="git ref to diff against (default: working-tree changes)")
ap.add_argument("--json", action="store_true", help="emit JSON only")
args = ap.parse_args()
targets, mode = resolve_targets(args)
ranges = added_line_ranges(args.base) if mode.startswith("git-diff") else {}
findings: list[Finding] = []
for path in targets:
for f in scan_file(path):
f.in_diff = in_ranges(f.line, ranges.get(path, [])) if ranges else True
findings.append(f)
# Diff-touching findings first, then by confidence.
rank = {"high": 0, "medium": 1, "low": 2}
findings.sort(key=lambda f: (not f.in_diff, rank.get(f.confidence, 9), f.file, f.line))
by_conf = {"high": 0, "medium": 0, "low": 0}
by_class: dict[str, int] = {}
for f in findings:
by_conf[f.confidence] = by_conf.get(f.confidence, 0) + 1
by_class[f.klass] = by_class.get(f.klass, 0) + 1
report = {
"scope": {"mode": mode, "files_scanned": len(targets), "files": targets},
"summary": {"by_confidence": by_conf, "by_class": by_class, "total": len(findings)},
"findings": [f.to_dict() for f in findings],
}
if args.json:
print(json.dumps(report, indent=2))
return 0
# Human-readable to stderr, JSON to stdout — easy to pipe or read.
if not targets:
print("scan_static: no scannable changed files found "
"(pass paths explicitly, or check you're in a git repo).", file=sys.stderr)
else:
print(f"scan_static: {len(targets)} file(s), {len(findings)} candidate(s) "
f"[{by_conf['high']} high / {by_conf['medium']} medium / {by_conf['low']} low]",
file=sys.stderr)
for f in findings:
tag = "diff" if f.in_diff else "ctx "
print(f" [{f.confidence:<6}] {tag} {f.file}:{f.line} {f.klass} ({f.catalog_id})",
file=sys.stderr)
print(json.dumps(report, indent=2))
return 0
if __name__ == "__main__":
sys.exit(main())