Code Quality Beyond Test Coverage
Most teams track code coverage. A report says 92% and everyone feels fine.
But coverage answers one question only. Did the tests run this code?
It doesn't say whether the tests would catch a bug. It doesn't say how hard the code is to change. And it can't see that the same business rule is copied in five places.
So this post looks at nine numbers together:
- Coverage. Did the tests run the code?
- Mutation score. Would the tests notice a bug?
- Cyclomatic complexity. How many paths run through the code?
- Cognitive complexity. How hard is the code to read?
- CRAP score. Is the code complex and untested at the same time?
- Hotspots. Is it complex and changing every week?
- Duplication. Is the same knowledge written in two places?
- Coupling. How many modules break if this one changes?
- Code smells. Which rules of thumb does the code break?
Each one is worked out by hand on one small JavaScript function. Then the tool runs on the same function and the two numbers are compared. When they differ, the post says why. Nothing here needs a tool to understand. The tools only make it fast.
Near the end there's a short section on what changes when a model writes most of the code. Not much, but the order of what to look at first does.
The function we'll measure
Here's the whole thing.
function calculateDiscount(age, premiumMember, orderValue) {
if (age < 0 || orderValue < 0) {
throw new Error('invalid input');
}
let discount = 0;
if (premiumMember) {
discount += 10;
}
if (age >= 60) {
discount += 5;
}
if (orderValue > 10000) {
discount += 10;
}
return Math.min(discount, 20);
}Ten statements: the input check, the throw, the let, three ifs with their three additions, and the return. One || inside the first check. A cap of 20 at the end. It only matters for a premium senior with a big order, because 10 + 5 + 10 is 25.
And here are the tests the team starts with. They use Node's built-in test runner, so there's nothing to install.
// tests/before.test.js
const { test } = require('node:test');
const assert = require('node:assert');
const { calculateDiscount } = require('../pricing.js');
test('premium member gets 10', () => {
assert.strictEqual(calculateDiscount(30, true, 5000), 10);
});
test('senior gets 5', () => {
assert.strictEqual(calculateDiscount(70, false, 1000), 5);
});
test('runs without error', () => {
calculateDiscount(30, true, 5000);
});The tests live in a tests folder next to pricing.js. The third test has no assertion. It comes back later.
The function looks harmless. Let's take the nine questions one by one.
Coverage: which statements did the tests run?
Coverage counts how much of the production code runs while the tests run. The usual forms are line coverage, branch coverage and function coverage.
By hand. List the ten statements and tick the ones that tests 1 and 2 execute.
Test 1 is a 30-year-old premium member with an order of 5,000. It runs the input check, sets discount to 0, adds 10 for premium, skips the age check and skips the big-order check. Test 2 is a 70-year-old with an order of 1,000. It adds 5 for age.
Two statements never run. The throw never runs, because no test sends bad input. The discount += 10 for big orders never runs, because no order goes above 10,000.
Statement coverage = executed statements / all statements
= 8 / 10
= 80%Branch coverage is stricter. Every if has two outcomes, true and false, and both count. The count below uses the four ifs only. A stricter count would also add the two halves of the ||.
if (invalid input) true: never false: ran
if (premiumMember) true: ran false: ran
if (age >= 60) true: ran false: ran
if (orderValue > 10000) true: never false: ran
Branch coverage = 6 / 8 = 75%With the tool. Node has coverage built in. This is Node 22.
$ node --test --experimental-test-coverage tests/before.test.js
file | line % | branch % | funcs % | uncovered lines
pricing.js | 77.78 | 66.67 | 100.00 | 3-4 13-14The uncovered lines are the same two places. Line numbers count from the top of the file, so 3 and 4 are the throw and its closing brace. The percentages are lower than the hand count, and here is why.
- Lines. Node counts every line in the file, including closing braces and the
module.exportsline. So it reports 14 of 18. - Branches. Node doesn't count if outcomes. It counts the code blocks that the V8 engine tracks: the whole file, the function body and each
ifblock. That's six here, and four of them were entered.
The hand count asks about both outcomes of every if. That's stricter and more useful. The gaps are the same two places either way.
The no-assert trap. Now run only the third test, the one with no assertion.
file | line % | branch % | funcs % | uncovered lines
pricing.js | 66.67 | 40.00 | 100.00 | 3-4 10-11 13-14Two thirds of the lines are "covered" by a test that checks nothing. It would pass if the function returned 999. The branch column says 2 of 5 now, because V8 folded the premium block into the function body when both ran exactly once. Coverage tells you what the tests touched, not what they checked.
That's what mutation testing is for.
Mutation score: would the tests notice a bug?
Mutation testing asks a simple question. If a small bug were put into the code, would a test fail?
The tool makes a copy of the code with one small change, called a mutant, and runs the tests against it.
- Killed. At least one test failed. The tests noticed the change. Good.
- Survived. Every test still passed. The tests can't tell the mutant from the real code. That needs a look.
By hand. Pick a few likely mutants and run the three tests in your head.
| Mutant | Test 1 (30, premium, 5000) | Test 2 (70, not premium, 1000) | Result |
|---|---|---|---|
age >= 60 becomes age > 60 | 10, passes | 70 is still over 60, passes | survived |
orderValue > 10000 becomes >= 10000 | 5000 is below both, passes | passes | survived |
big-order discount += 10 becomes -= 10 | never runs, passes | never runs, passes | survived |
|| becomes && | no throw either way, passes | passes | survived |
age < 0 becomes age <= 0 | 30 is above 0, passes | passes | survived |
premium discount += 10 becomes -= 10 | returns -10, fails | killed | |
error message becomes "" | never runs, passes | passes | survived |
Six of seven survive. Test 1 does real work, because it checks an exact value. Test 2 checks a value too, but 70 is so far from the boundary at 60 that it can't tell >= from >.
Mutation score = killed mutants / all mutants
= 1 / 7
= 14%With the tool. Stryker is the mutation tester for JavaScript. It makes far more mutants than a person would. Thirty, for this one function.
Many are ones a person wouldn't bother with. Every condition forced to true and to false, every block emptied, Math.min turned into Math.max. Test 1 kills most of those, which is why the tool's score comes out higher than the hand list.
$ npx stryker run
File | % Mutation score | # killed | # survived
pricing.js | 56.67 | 17 | 13Thirteen survivors. Every one of them is a test nobody wrote. Reading the survivor list is more useful than the score. Here are two of them, exactly as Stryker prints them.
[Survived] EqualityOperator
pricing.js:9:7
- if (age >= 60) {
+ if (age > 60) {
[Survived] LogicalOperator
pricing.js:2:7
- if (age < 0 || orderValue < 0) {
+ if (age < 0 && orderValue < 0) {Fixing it. Each survivor names its own test.
- The boundary at 60 needs a test at 60 and one at 59.
- The boundary at 10,000 needs 10,000 and 10,001.
- The cap needs a customer who would get more than 20 without it, so a premium senior with a big order.
- The input check needs a negative age and a negative order. It also needs a zero for each, because zero is where
<and<=disagree.
test('age 60 is a senior', () => {
assert.strictEqual(calculateDiscount(60, false, 1000), 5);
});
test('age 59 is not a senior', () => {
assert.strictEqual(calculateDiscount(59, false, 1000), 0);
});
test('big order gets 10', () => {
assert.strictEqual(calculateDiscount(30, false, 10001), 10);
});
test('order of exactly 10000 does not', () => {
assert.strictEqual(calculateDiscount(30, false, 10000), 0);
});
test('discount is capped at 20', () => {
assert.strictEqual(calculateDiscount(65, true, 20000), 20);
});
test('negative age is rejected', () => {
assert.throws(() => calculateDiscount(-1, false, 100));
});
test('negative order value is rejected', () => {
assert.throws(() => calculateDiscount(30, false, -1));
});
test('age 0 and order 0 are allowed', () => {
assert.strictEqual(calculateDiscount(0, false, 0), 0);
});Run Stryker again.
File | % Mutation score | # killed | # survived
pricing.js | 96.67 | 29 | 1The one survivor changes the error message to an empty string. No test reads the message. That's a survivor you can accept, and it's a good example of why the score should never be forced to 100. Read each survivor and decide. The score by itself isn't the goal.
With these tests, coverage went from 78% to 100% and the mutation score went from 57% to 97%. The first number moved a little. The second moved a lot, and it's the one that tells you about the tests.
For how coverage and mutation score behave across a whole team, the engineering metrics course has a chapter on test coverage and effectiveness.
Cyclomatic complexity: how many paths?
Cyclomatic complexity counts the independent paths through a function. Every decision adds one.
By hand, the quick way. Count the decision points and add one.
if (age < 0 || orderValue < 0) -> 1 for the if, 1 for the ||
if (premiumMember) -> 1
if (age >= 60) -> 1
if (orderValue > 10000) -> 1
1 + 5 = 6The || counts because it's a decision too. If age < 0 is true, the second half is never looked at.
By hand, the original way. Thomas McCabe defined the number in 1976 from the control-flow graph. Draw every statement as a node and every possible jump as an edge.
M = E - N + 2
= 15 edges - 11 nodes + 2
= 6Same answer. The quick way is what tools actually do. The graph is why the quick way works.
With the tool. ESLint has a complexity rule built in. Set the limit to 0 and it reports every function.
$ npx eslint pricing.js
1:1 Function 'calculateDiscount' has a complexity of 6. Maximum allowed is 0ESLint (version 10 here) also counts things a hand count can miss. A default parameter value, an optional chain ?., a ?? and a ternary each add one. All of them are small hidden decisions.
Reading the number. McCabe suggested 10 as the limit for one function. A rough scale:
| Complexity | What it usually means |
|---|---|
| 1 to 5 | simple |
| 6 to 10 | fine, but read it |
| 11 to 20 | needs a careful review |
| 21 to 50 | complex, split it |
| over 50 | very hard to test at all |
These aren't laws. The number is most useful when it grows. A function at 8 last quarter and 19 today is telling you something.
Cognitive complexity: how hard is it to read?
Cyclomatic complexity counts paths. Cognitive complexity asks a different question. How much does a reader have to remember at once?
Look at these two functions. Both do the same job.
function canProcessNested(user, account) {
if (user.isActive()) {
if (user.hasSubscription()) {
if (!user.isSuspended()) {
if (account.hasCredit()) {
return true;
}
}
}
}
return false;
}function canProcessFlat(user, account) {
if (!user.isActive()) return false;
if (!user.hasSubscription()) return false;
if (user.isSuspended()) return false;
if (!account.hasCredit()) return false;
return true;
}The second shape is called guard clauses. Each if checks one failure case and returns at once, so nothing is nested.
Cyclomatic complexity is 5 for both. Four ifs plus one. But the first one is tiring. Every level is one more condition the reader has to remember while reading the next line.
By hand. SonarSource published the rules in a white paper called Cognitive Complexity, written by G. Ann Campbell and first released in 2016. There are three rules:
- Add 1 for each break in the straight-line flow. An
if, anelse, a loop, acatch, aswitch, a ternary, or a run of&&or||. - Add 1 more for each level of nesting that the break sits inside.
- Add nothing for shorthand that makes code shorter, like a method call or an early
return.
Nested version Flat version
if (isActive) +1 if (!isActive) +1
if (hasSubscription) +1 nesting +1 if (!hasSub) +1
if (!isSuspended) +1 nesting +2 if (isSuspended) +1
if (hasCredit) +1 nesting +3 if (!hasCredit) +1
= 10 = 4Same paths. One number is 10, the other is 4. That gap is the reason the metric exists.
For calculateDiscount, the four ifs sit at the top level, so there's no nesting charge. The paper adds 1 for the || run, which gives 5.
With the tool. The JavaScript rule lives in eslint-plugin-sonarjs.
Refactor this function to reduce its Cognitive Complexity from 4 to the 0 allowed calculateDiscount
Refactor this function to reduce its Cognitive Complexity from 10 to the 0 allowed canProcessNested
Refactor this function to reduce its Cognitive Complexity from 4 to the 0 allowed canProcessFlatThe nested and flat versions match the hand count exactly. calculateDiscount comes out as 4, not 5. I read the rule's source to find out why.
The plugin (version 4.2) charges for runs of && but not for runs of || or ??. So the paper says 5 and the plugin says 4. Either way it's low. The gap between 10 and 4 is what to look at.
SonarSource flags a function at 15. Teams tune that, but 15 is a reasonable start.
CRAP: complex and untested at the same time
CRAP stands for Change Risk Anti-Patterns. Alberto Savoia and Bob Evans defined it in 2007 for a Java tool called crap4j. It joins two numbers you already have.
CRAP = complexity² × (1 - coverage)³ + complexityComplexity is cyclomatic complexity. Coverage is the coverage of that one function, as a fraction from 0 to 1.
By hand. Our function has complexity 6, so complexity squared is 36.
| Coverage | 36 × (1 - coverage)³ | + 6 | CRAP |
|---|---|---|---|
| 0% | 36 × 1 = 36 | + 6 | 42 |
| 50% | 36 × 0.125 = 4.5 | + 6 | 10.5 |
| 80% (the first tests) | 36 × 0.008 = 0.29 | + 6 | 6.3 |
| 100% (after the boundary tests) | 36 × 0 = 0 | + 6 | 6 |
Now the same table for a function of complexity 15.
| Coverage | 225 × (1 - coverage)³ | + 15 | CRAP |
|---|---|---|---|
| 0% | 225 | + 15 | 240 |
| 50% | 28.1 | + 15 | 43.1 |
| 80% | 1.8 | + 15 | 16.8 |
| 100% | 0 | + 15 | 15 |
The cube is what makes the number useful. Going from 50% to 80% coverage on the complex function drops CRAP from 43 to 17. But once coverage is high, the only way down is to lower the complexity.
Adding tests moves a function along its curve, to the right and down. Lowering complexity moves it onto a lower curve.
Reading the number. crap4j called anything over 30 "crappy", and 30 is still the usual line. The bands below are mine, only the 30 comes from crap4j. A function can never score below its own complexity, so 6 is the floor for our function.
| CRAP | Meaning |
|---|---|
| under 6 | low change risk |
| 6 to 15 | usually fine |
| 15 to 30 | worth a look |
| over 30 | complex and unprotected, fix before touching |
With the tool. There isn't a popular CRAP tool for JavaScript, and that's fine. You already have both inputs. ESLint gives complexity per function. Node's coverage gives coverage per file, and c8, a coverage tool for Node, shows it per function with --reporter=html. One line of arithmetic gives CRAP.
Hotspots: complex and changing every week
CRAP pairs complexity with test coverage. Hotspot analysis pairs complexity with a different number. How often does this file change?
The idea comes from Adam Tornhill's book Your Code as a Crime Scene (2015). Complex code that nobody touches is a problem that can wait. Complex code that changes every week is where the bugs are being made right now.
By hand. Say the shop is a small online store with five modules: checkout, pricing, shipping, invoice and email. Its pricing.js is the file we've been measuring. Git already has the change count. One command lists the files that changed most in the last 90 days.
$ git log --since='90 days ago' --format=format: --name-only | grep . | sort | uniq -c | sort -rn | head
41 src/checkout.js
29 src/pricing.js
27 src/email.js
9 src/invoice.js
2 src/shipping.jsNow put the complexity of each file next to it. ESLint's complexity rule gives it per function. Add the functions up per file.
| File | Changes in 90 days | Complexity | Churn × complexity |
|---|---|---|---|
| checkout.js | 41 | 38 | 1,558 |
| invoice.js | 9 | 22 | 198 |
| pricing.js | 29 | 6 | 174 |
| email.js | 27 | 3 | 81 |
| shipping.js | 2 | 6 | 12 |
Read the whole table. Email changes a lot, but it's simple, so leave it. Invoicing is complex, but nobody touches it, so it can wait. Checkout is complex and changes every second day. That's the hotspot, and it's where the next bug will come from.
With the tool. Tornhill's open-source tool is code-maat, and CodeScene is the hosted version. For one repo the shell pipe above is enough.
Duplication: the same knowledge in two places
Duplication measures how much code appears more than once.
Say the team adds shipping discounts quickly and copies the function.
// shipping.js
function calculateShippingDiscount(age, premiumMember, orderValue) {
if (age < 0 || orderValue < 0) {
throw new Error('invalid input');
}
let discount = 0;
if (premiumMember) {
discount += 10;
}
if (age >= 60) {
discount += 5;
}
if (orderValue > 5000) {
discount += 15;
}
return Math.min(discount, 30);
}By hand. Put the two files side by side and count the identical lines. Whole lines 2 to 11 match. That's 10.
Line 1 shares the parameter list and line 12 shares if (orderValue >. A tool that compares tokens instead of whole lines counts those two half lines too, and gets 11. Each file is 18 lines with its blank line and its module.exports, so 36 in total.
Duplication = duplicated lines / total lines
= 11 / 36
= 31%With the tool. jscpd finds copied blocks. By default it ignores repeats shorter than 5 lines.
$ npx jscpd --min-lines 5 --min-tokens 30 pricing.js shipping.js
Clone found (javascript)
- pricing.js [1:27 - 12:19] (12 lines, 58 tokens)
shipping.js [1:35 - 12:19]
Duplicated lines: 11 (30.56%)The clone runs from column 27 of line 1 to column 19 of line 12. Those are the two half lines counted above. 11 lines, 31%.
Same knowledge, or same shape? Not every repeated line is a problem. The real test is whether the two copies are the same knowledge.
The input check and the premium and senior rules are one business rule in both files. When the senior age changes to 65, someone has to remember both places, and one day someone won't.
The thresholds at the bottom are different rules that happen to look alike. Pull the shared rule into one function and leave the thresholds where they are.
Coupling: how many modules break if this one changes?
Every number so far looked inside one function. Coupling looks at the lines between files.
Robert C. Martin defined two counts for a module in 1994, and they're still the ones people use:
- Afferent coupling, Ca. How many other modules depend on this one. Count the files that import it.
- Efferent coupling, Ce. How many modules this one depends on. Count its own imports.
From those two comes instability:
I = Ce / (Ca + Ce)An I of 0 means the module depends on nothing, while other modules depend on it. It's stable, and changing it is risky. An I of 1 means it depends on others and nothing depends on it. It can change freely.
By hand. Take five modules from the shop. Count the require lines.
| Module | Imported by (Ca) | Imports (Ce) | I = Ce / (Ca + Ce) |
|---|---|---|---|
| pricing.js | checkout, shipping, invoice = 3 | 0 | 0 / 3 = 0, stable |
| email.js | checkout = 1 | 0 | 0 / 1 = 0, stable |
| shipping.js | checkout = 1 | pricing = 1 | 1 / 2 = 0.5 |
| invoice.js | checkout = 1 | pricing = 1 | 1 / 2 = 0.5 |
| checkout.js | 0 | pricing, shipping, invoice, email = 4 | 4 / 4 = 1, unstable |
Three modules break if calculateDiscount changes its behaviour. That's exactly the module that needs a CRAP of 6 and a mutation score of 97%, which it now has.
Stable and well tested is the right pair. Stable, complex and weakly tested is the dangerous one. It's also common. The module everybody depends on is usually the one nobody wants to change.
Checkout, at the other end, can change every day without breaking anyone. That matches the hotspot table. It's fine for the unstable module to be the busy one. It's a problem when the stable one is.
With the tool. madge prints the import graph for a JavaScript folder as JSON. Count the entries in each list for Ce, and count how often a name appears across all lists for Ca.
$ npx madge --json src/
{
"checkout.js": ["email.js", "invoice.js", "pricing.js", "shipping.js"],
"email.js": [],
"invoice.js": ["pricing.js"],
"pricing.js": [],
"shipping.js": ["pricing.js"]
}dependency-cruiser does the same with rules you can fail the build on.
Code smells: the rules a linter counts
A code smell isn't a bug. It's a shape that usually makes code harder to change later. There's no formula. Static analysis tools apply a list of rules, and each rule has a number behind it.
By hand. These six catch most of what matters, and you can check them in a code review without any tool.
| Smell | Number to check | Why it hurts |
|---|---|---|
| Long parameter list | more than 4 parameters | callers get the order wrong |
| Long function | more than 50 lines | it does more than one job |
| Deep nesting | more than 3 levels | cognitive complexity climbs fast |
| Boolean flag argument | any true, false, true call | nobody can read the call site |
| Empty catch | any | errors disappear |
| Magic number | any unexplained literal | 10000 means nothing next month |
Here's the boolean flag one, because it's in every codebase.
generateReport(data, true, false, true);What do the booleans mean? Nobody knows without opening the function.
generateReport(data, {
includeCharts: true,
includeRawData: false,
compress: true,
});Both versions work. Only one of them can be read.
With the tool. ESLint ships max-params, max-lines-per-function, max-depth, no-empty and no-magic-numbers. eslint-plugin-sonarjs adds a few hundred more. A report says "Code smells: 143" and that number means very little by itself. A five-million-line codebase will always have more than a twenty-thousand-line one.
Measure it like this instead:
- Smells per 1,000 lines. Comparable across projects.
- New smells in this pull request. The only number a reviewer can act on.
- Smells in the hotspot files. That's where they cost the most.
Numbers you read but don't compute
Four more show up on dashboards. They're worth understanding, but there's nothing to learn from working them out by hand.
- Maintainability Index. An old formula that mixes Halstead volume, cyclomatic complexity and lines of code into one score, usually scaled 0 to 100. Every tool computes it a bit differently, so only compare scores from the same tool. Use it to find the worst module, not to say one module is 12% better than another.
- Halstead metrics. Counts of operators and operands from the 1970s. Almost nobody reads them directly. They live on as an input to the Maintainability Index.
- Defect density. Confirmed bugs per thousand lines of code. It needs bug data, not code, and it depends on how hard the team looks. Good for a trend inside one team. Useless for comparing companies.
- Technical debt. Tools estimate it by adding up a repair time per finding. Two minutes for an unused import, thirty for a complex method, and so on, until the report says "11 days". Treat it as an indicator. A tool can't see that five services carry the same pricing rule because the team boundaries were drawn wrong, and that debt is the expensive kind.
Reading all nine together
No single number says whether code is good. Here's the whole set, one line each.
| Number | Question it answers | By hand | Tool |
|---|---|---|---|
| Coverage | Did the tests run it? | executed / all statements | node, c8 |
| Mutation score | Would the tests notice a bug? | killed / all mutants | Stryker |
| Cyclomatic | How many paths? | decisions + 1 | ESLint complexity |
| Cognitive | How hard to read? | breaks + nesting | sonarjs |
| CRAP | Complex and untested? | c² × (1 - cov)³ + c | arithmetic |
| Hotspots | Complex and busy? | git changes × complexity | code-maat |
| Duplication | Same knowledge twice? | copied / all lines | jscpd |
| Coupling | Who breaks if it changes? | Ce / (Ca + Ce) | madge |
| Smells | Which rules broken? | six checks per review | ESLint |
And here's what the usual combinations mean.
| Coverage | Mutation | Complexity | Likely meaning |
|---|---|---|---|
| 95% | 35% | low | tests run the code and check almost nothing |
| 95% | 90% | low | strong tests on simple code, leave it alone |
| 90% | 85% | high | well tested but expensive to change, simplify next |
| 40% | 30% | high | the risky part of the codebase |
| 60% | 80% | low | gaps in coverage, but what's tested is tested well |
A report, read properly. Say a service shows this.
Line coverage: 91%
Branch coverage: 76%
Mutation score: 48%
Cyclomatic, average: 8
Cognitive, average: 12
Duplication: 3.5%
Critical static issues: 0
Methods with CRAP > 30: 17The first line looks great. The rest tells a different story. A mutation score of 48% means about half of the small changes the tool tried got past the tests. Seventeen methods are both complex and unprotected.
Asking the team to push coverage from 91% to 95% would change nothing. This would:
- List the 17 methods with CRAP over 30, highest first.
- Cross that list with the hotspot list. Start where both agree.
- Read the surviving mutants inside those methods.
- Write boundary tests for each survivor that matters.
- Split the methods with the highest cognitive complexity.
- Run mutation testing again on those files.
- From here on, track new code only. The legacy total won't move for months.
What to gate
A gate is a check that has to pass before code merges. You don't need a gate for every number. A reasonable starting gate for a JavaScript service:
New code line coverage >= 80%
New code branch coverage >= 70%
Critical static issues = 0
New duplication < 3%
Cognitive complexity per fn <= 15
CRAP > 30 reviewed, not blocked
Surviving mutants, changed files reviewed
Hotspot files mutation testing required
Changed lines per pull request <= 400
New dependencies listed in the PR and checked
Unused exports 0 newMutation testing on every commit is slow on a big codebase. The usual shape is:
Pull request
-> unit tests + coverage + ESLint
-> Stryker on changed files only
Nightly
-> Stryker on the whole pricing / payments / auth modulesA pricing engine deserves stricter gates than generated mapping code. Set the gate per module.
When a model writes most of the code
The nine numbers don't change when the code comes from a model. What changes is which ones move first.
- Mutation score matters more. Ask a model for tests and it often writes tests that assert whatever the code does today. They pass and raise coverage. They catch nothing. The survivor list finds them in one run.
- Duplication matters more. A model doesn't know that
calculateDiscountalready exists in another folder. It writes a new one. - Cognitive complexity matters more. Generated code nests deeply, because the model never has to read it back.
Three more counts are worth adding, and each one is a single command.
Change size
Generated pull requests are big, and a reviewer tends to read the first part carefully and skim the rest. A 2006 SmartBear study of code review at Cisco found that reviewers find far fewer defects once a review goes past about 400 lines. Count it before asking for a review.
$ git diff --stat main...HEAD | tail -1
14 files changed, 1240 insertions(+), 96 deletions(-)Add both numbers. 1,240 plus 96 is 1,336 changed lines, more than three times the limit. Split the pull request before you ask for a review.
Dead code
A model writes helpers for cases nobody asked for. They compile and get exported. Nobody calls them. By hand, take each exported name and search the codebase for it.
In the shop from the coupling section, say checkout stops using shipping. The search comes back empty, and so does the tool.
$ npx knip
Unused files (1)
shipping.jsknip also lists unused exports and unused dependencies. Run it on every generated pull request.
Dependencies added
A model adds a package where a person would write ten lines. Count the new entries in package.json per pull request. Then check each one exists, is maintained, and is the package you meant.
Models have suggested package names that were never published. People have then registered those names with malware inside. Security researchers call this slopsquatting.
$ npm view <package> version time.modifiedAn old modified date, or no package at all, is a reason to stop.
A prompt that carries the gates
The gates above only help if the model is held to them on every task. So put the gates and the validation into the prompt itself. Then the model runs the checks before it says done, and it reports numbers instead of adjectives.
The rules don't depend on the language, and the model picks the tools. It reads the repository and uses the checks the project already has. It only uses the list at the end of the prompt when a check is missing.
Keep the prompt in the file your coding tool reads on every task, such as CLAUDE.md or AGENTS.md.
You are changing code in this repository. Do the task below, then prove it meets the gates.
TASK
<one or two sentences: what to change, and why>
BEFORE WRITING CODE
- Search the repo for an existing function that already does this. Reuse it. Do not write a second one.
- Keep the change under 400 lines. If the task needs more, stop and propose a split.
- Do not add a dependency. If you believe one is needed, name it, give its registry page, its last release date, and why ten lines of code will not do. Wait for approval.
WHILE WRITING CODE
- No function above cyclomatic complexity 10 or cognitive complexity 15. Use guard clauses, not nesting.
- No new parameters beyond 4. Use an options or parameter object.
- No boolean flag arguments, no empty catch blocks, no unexplained numbers.
- Every public function must have a caller. Do not leave helpers "for later".
TESTS
- Each test name states the rule it checks, and asserts an exact expected value. Never assert whatever the code currently returns.
- Test every boundary on both sides: if the rule is >= 60, test 59 and 60. If it is > 10000, test 10000 and 10001.
- Test the failure paths: invalid input, zero, empty, the cap.
VALIDATION (run these, paste the real output, do not summarise it)
First detect the language and the tools this repository already uses (package.json, pyproject.toml, pom.xml,
go.mod, CI config, lint config). Use the project's own commands where they exist. Where a check is missing,
pick the tool for the language from the TOOLS list below, install it as a dev dependency, and say so.
State the exact command you ran before each output.
1. coverage -> line and branch % for the files you changed
2. complexity -> 0 functions over the limits, with cyclomatic and cognitive rules on
3. mutation -> mutation score for the changed files, and the full survivor list
4. duplication -> 0 new clones of 5 lines or more
5. dead code -> 0 unused functions, files or dependencies that you added
6. git diff --stat -> total changed lines
TOOLS (use when the project has nothing for that check)
JavaScript / TypeScript: node --test --experimental-test-coverage or c8; ESLint complexity + eslint-plugin-sonarjs; Stryker; jscpd; knip
Python: pytest --cov --cov-branch; radon cc + complexipy; mutmut; pylint duplicate-code or jscpd; vulture
Java / Kotlin: JaCoCo; PMD CyclomaticComplexity + CognitiveComplexity; PIT; PMD CPD; PMD unused rules
Go: go test -cover; gocyclo + gocognit; gremlins; dupl; staticcheck
Any other language: jscpd covers duplication for over a hundred languages; find the rest for that language, name them, and ask before installing
REPORT (fill this in, then stop)
- Changed lines:
- Branch coverage of changed files:
- Mutation score of changed files, and for each survivor: the mutant, and either the test you added or the reason it is acceptable
- Highest cyclomatic and cognitive complexity in the change, and which function
- New dependencies: none, or the approved list
- Anything you could not verify, and why
If any gate fails, fix the code and run the validation again. Do not lower a threshold. Do not say "done" without the report.Three things make this prompt work.
The model finds its own tools. It reads the repository first, so it runs the project's real test and lint commands, not guesses. The list at the end only fills gaps, and it has to say which tool it added.
It asks for pasted output. "Tests pass" is a claim. A pasted mutation score with the survivor list is evidence. A model can't hide that.
The survivor list is the review. Every survivor is either a missing test or a stated reason. That's the same judgement a person applies, and it's the part a coverage number skips.
What the numbers point at
Every one of these numbers is a symptom of a design principle being kept or broken. The next post takes seven change requests through the same shop and shows each principle at work. For now, the map:
- Duplication points at DRY, don't repeat yourself.
- Cognitive complexity points at KISS, keep it simple.
- Cyclomatic complexity points at single responsibility. A function with 20 paths is doing several jobs.
- Coupling points at high cohesion and low coupling. Keep what changes together in one module, and keep the lines between modules few. A module with an instability of 0 and 40 dependents should be small.
- Hotspots point at separation of concerns, one job per file. A file that changes for pricing reasons, email reasons and layout reasons is doing three jobs.
- Long parameter lists and flag arguments point at the wrong level of abstraction, a function that makes its caller know too much.
- Surviving mutants and dead code point at YAGNI, you aren't going to need it. If no test can tell the difference and nobody calls it, does it need to exist?
Nine good numbers on a dashboard were never the goal. The goal is one practical question. How safely can we understand, test, change and run this code?