What your document score means

The short version

Every document starts at 100. Each issue Qrtr finds takes something off, and how much it takes off depends on how serious the issue is. The result is a number between 0 and 100.

100 means one thing only: Qrtr found nothing. It is not a rounding of "very good" — a single minor issue takes a document to 99. If you are looking at 100, the document satisfied every requirement that was checked, with nothing outstanding.

Issues you have dismissed, or marked as does not apply, are excluded before the score is calculated. Clearing something that genuinely does not apply is not a way of hiding a problem; it removes a finding that should not have counted in the first place.

Two things follow from how the arithmetic works, and they are the two things worth knowing:

• Severity matters far more than count. One critical issue costs what dozens of minor ones cost.
• The scale compresses as it falls. The same issue costs fewer points on an already-low score than on a high one, so the first fixes on a good document move the number most.

That is deliberate. A document with a lot of small problems is untidy. A document with a serious problem in it is a different kind of document, and the score is built to say so.

This applies to everything Qrtr surfaces, not only requirement findings. Every issue it raises — of whatever type — carries a severity, and they all feed the same calculation.

The five severity levels, and the three you can see

Qrtr grades every issue on a five-point severity scale internally, whatever kind of issue it is, and the report groups them into three bands:

This grouping matters more than it looks. Two issues sitting in the same band can cost very different amounts — a severity 5 costs three times what a severity 1 costs at the top of the Minor band, and 50% more than a severity 4 despite both showing as Critical. If two documents show the same three band counts and score differently, this is why.

What one of each is worth

The clearest way to feel the weighting is to ask how many minor issues it takes to cost the same as one serious one. Against the lightest minor issue:

Sixty-three. If you take one thing from this article, take that.

Worked examples

Here is what actual documents look like. The band counts are what you would see in the report.

Note the third row against the last. The document in trouble has less than twice the minor issues of the typical first draft, but four times the critical ones — and that is what drives it from 84 down to 62. Severity, not volume.

Which fix moves the number

Take the typical first draft above — 12 severity 1, 6 severity 2, 3 severity 3, 1 severity 4, scoring 84. Fix exactly one issue and re-check:

Fixing a single minor issue does not move the score at all. Fixing the one critical issue moves it three points, in less time.

This is the practical lesson of the whole article: work top down. If you are trying to lift a score, the queue is already sorted for you by severity, and starting at the top is not just good practice, it is arithmetically the only thing that works quickly.

To make the point harder — from that same document, fixing all twelve minor issues takes the score from 84 to 85. Fixing the single critical one takes it from 84 to 87.

Why a bad document does not simply score zero

This is a deliberate design decision rather than a side effect, and it is worth understanding because it changes how the score behaves when things are going badly.

The penalties are combined and then curved, rather than subtracted directly. Each further penalty removes a proportion of what is left rather than a fixed amount, so the score falls steeply at first and then flattens out. It approaches zero without a cliff: it takes something like a hundred severity 5 issues before the display actually reads 0, and fifty of them still shows 1.

The reason is that a score which bottoms out at zero stops being useful exactly when it is most needed. A team working through a badly failing document would see 0, fix ten serious problems, and still see 0 — which tells them nothing and is quietly demoralising. Under this curve the number moves as they work, so progress is visible from the first fix onward, even while the document is still a long way from passing.

The calculation

The full arithmetic, for anyone who wants it.

Count the issues at each severity level — call them n1 through n5 — and compute a penalty:

The exponents control how each severity accumulates *within the penalty*. Below 1 — severities 1 and 2 — each additional issue adds slightly less penalty than the last. At exactly 1, severity 3, every issue adds the same 3.5. Above 1 — severities 4 and 5 — each additional issue adds slightly more than the last, so a pile-up of critical findings compounds.

The division by 150 sets how quickly the curve falls, and the exponential is what keeps the score from hitting a floor.

Penalty points and score points are not the same thing

This is worth separating out, because the two move in opposite directions and it is easy to draw the wrong conclusion.

The penalty a critical issue adds does grow with each one: 9.0 for the first, 12.4 for the tenth. But the score is the *curve* of the penalty, and the curve flattens. Once a document is already low, every additional issue of any kind costs fewer actual score points than it would have on a healthy document.

Here is the score cost of the nth critical issue, on a document that has nothing else wrong with it:

The compounding wins briefly — the second critical costs more than the first — and then the flattening takes over. So critical issues bite hardest early, and keep biting further down the scale than minor ones do, but nothing costs much once a document is already in the twenties.

The practical reading is the same either way: the points are in the serious issues, and they are worth most while the document is still in reasonable shape.

You are welcome to reproduce this. It is published deliberately: a score you cannot audit is a score you should not have to trust.

To see the flattening in practice:

A very bad document lands in single figures rather than at zero. Do not read the difference between 7 and 3 as meaningful — anything that low is telling you the same thing. What matters at that end of the scale is that the number is still moving.

One caveat on watching the dial

The score is displayed as a whole number, rounded down. Real movement smaller than a point therefore will not show. Fixing a single severity 1 issue in the worked example above genuinely improves the score, from 84.29 to 84.38 — but both display as 84.

So the number does move as you work, and on any meaningful fix you will see it. If you have cleared one small thing and the display has not budged, the fix still counted.

What the score does not account for

Three things worth being clear about, because they cause arguments.

Length. The score counts issues, not issues per page. Five problems in a three-page memo and five in a three-hundred-page report produce the same number. A long document is not penalised for being long, but nor does it get credit for the proportion of it that was fine.

Substance. The score reflects what Qrtr can check — whether requirements are satisfied, whether citations hold up, and the other issue types it surfaces. It does not tell you whether the reasoning holds, whether the recommendation is right, or whether someone with twenty years in the discipline would put their name to it. That is what the review workflow is for.

Comparability. A score is only comparable against another score produced by the same quality profile. Switch a requirement off, replace a requirements document or change the threshold and later checks are measuring against a different yardstick. 

The score and the pass threshold

The score on its own is not the verdict. Your project's quality profile carries a pass threshold, and the verdict banner compares the score against it. 

And a Qrtr Check passing score is still provisional until a person approves it.