Content Audit Playbook

Governance metadata ｜ Owner: Knowledge base administrator ｜ Last reviewed: 2026-10-28 ｜ Next review: 2027-01-31

The problem this solves: a knowledge base rots most quietly when "the structure is fine but the content is wrong" — links resolve, the build passes, the page exists, yet a section talks about something else or a number disagrees with its source. Automated checks cannot see this, and reading everything by hand does not scale.
The audit is split into four layers below. Each states what the machine can do, what a human must do, and gives a rerunnable command.

1. What the five gates actually cover

pnpm run check chains five scripts across structure, text, basis, arithmetic and components.

Gate	Script	Catches	Cannot catch
Text hygiene	`check-text-hygiene.mjs`	`~`/`_` separators in H1, stray escapes in code blocks, plain-text references, emoji in headings	Whether the content is correct
Consistency	`check-consistency.mjs`	Dead links, anchors, table column counts, tables split by blockquotes, self-links, duplicate body fingerprints, template-style checklists, ZH/EN structural drift	Whether statements are accurate
Metric authority	`check-metrics.mjs`	Threshold restatements outside the authoritative article	Whether the threshold itself is sensible
Number recomputation	`check-numbers.mjs`	Formula-versus-conclusion mismatches (105 assertions)	Whether the inputs are realistic
Component defaults	`check-component-defaults.mjs`	Tool component defaults drifting from the article baselines	Whether the tool logic is correct

Key point: these gates guarantee "does not contradict itself", never "is correct". The latter requires a human.

2. Four audit layers and their commands

Layer 1: heading versus body

What to check: whether the content a heading claims (how many, what subject) is actually delivered.

What the machine does: counts structure. If a heading says "six levers", count the H3s / bold list items / table rows in that section.

bash

# Spot-check quantity claims (129 headings contain "N items/steps/layers")
python3 - <<'PY'
import pathlib,re
CN={'一':1,'二':2,'三':3,'四':4,'五':5,'六':6,'七':7,'八':8,'九':9,'十':10}
QTY=r'([一二三四五六七八九十]|\d+)\s*(大|项|步|阶|层|类|种|档|级|条)'
for p in pathlib.Path('docs').rglob('*.md'):
    s=str(p)
    if '/en/' in s or '/public/' in s or '.vitepress' in s: continue
    inf=False
    for l in p.read_text(encoding='utf-8').split('\n'):
        if l.strip().startswith('```'): inf=not inf; continue
        if inf: continue
        if not l.startswith('## '): continue
        q=re.search(QTY,l)
        if q: print(f"{s[5:-3]}  {l[3:][:50]}  -> {q.group(1)}{q.group(2)}")
PY

What the human does: read each section's heading, first sentence and skeleton, and judge whether it stays on topic.

Lesson: three statistical approaches were tried and all failed —
Method Result Why it fails
Chinese bigram cosine similarity median 0.04 Headings are short, bodies long; literal overlap is naturally tiny
Heading-term hit rate in body 481 of 595 sections scored "0 hits" Tokenisation produced fragments like "global sales module sub-art"
Section terms assigned to business modules 208 sections "don't look like their module" Articles 01 and 02 are inherently cross-functional
Off-topic is a semantic question; literal statistics cannot detect it. This layer is human-only; the machine only narrows the field.

Method	Result	Why it fails
Chinese bigram cosine similarity	median 0.04	Headings are short, bodies long; literal overlap is naturally tiny
Heading-term hit rate in body	481 of 595 sections scored "0 hits"	Tokenisation produced fragments like "global sales module sub-art"
Section terms assigned to business modules	208 sections "don't look like their module"	Articles 01 and 02 are inherently cross-functional

Layer 2: table versus conclusion

What to check: whether the numbers a table produces actually support the qualitative verdict that follows it.

What the machine does: locates "table followed by a numeric conclusion" pairs.

bash

# Find the 44 "table followed by a numeric conclusion" spots
python3 - <<'PY'
import pathlib,re
pat=re.compile(r'(✅|❌|达标|通过|未达标|仍高|仍低|高于|低于|结论|因此)')
for p in pathlib.Path('docs').rglob('*.md'):
    s=str(p)
    if '/en/' in s or '/public/' in s or '.vitepress' in s: continue
    lines=p.read_text(encoding='utf-8').split('\n')
    inf=False; i=0
    while i<len(lines):
        l=lines[i]
        if l.strip().startswith('```'): inf=not inf; i+=1; continue
        if inf: i+=1; continue
        if l.strip().startswith('|') and i+1<len(lines) and re.match(r'^\s*\|[\s:|-]+\|\s*$',lines[i+1]):
            j=i+2
            while j<len(lines) and lines[j].strip().startswith('|'): j+=1
            for k in range(j, min(j+7,len(lines))):
                t=lines[k].strip()
                if t and pat.search(t) and re.search(r'\d', t):
                    print(f"{s[5:-3]}:{k+1}  {t[:100]}")
                    break
            i=j; continue
        i+=1
PY

What the human does: judge whether the conclusion and the table use the same basis.

Real traps from this library:
Symptom Reality
LTV:CAC of 1.45 labelled "⚠️ on the line" Against the 1.2 go-line it just clears; against the 3.0 steady-state line it is far short — two bases spoken of as one
Gift cap "no more than 5% of MAP" with the example USD 19.95 5% of MAP is 16.96; 19.95 is 5% of MSRP — the same gift pack flips verdict depending on basis
Margin 18.5% EBIT computed on net revenue 367.08, but the percentage divides by MSRP 399; on net revenue it is 20.1%

Symptom	Reality
LTV:CAC of 1.45 labelled "⚠️ on the line"	Against the 1.2 go-line it just clears; against the 3.0 steady-state line it is far short — two bases spoken of as one
Gift cap "no more than 5% of MAP" with the example USD 19.95	5% of MAP is 16.96; 19.95 is 5% of MSRP — the same gift pack flips verdict depending on basis
Margin 18.5%	EBIT computed on net revenue 367.08, but the percentage divides by MSRP 399; on net revenue it is 20.1%

Layer 3: cross-article number citations

What to check: when an article cites another's figure, is the figure right and is the basis carried along?

What the machine does: pins core numbers, then finds every citation.

bash

# Pin 11 core numbers and list which articles they appear in
python3 - <<'PY'
import pathlib,re
CORE={'73.69':'07-finance/07-06-pl-model','88.69':'07-finance/07-06-pl-model',
      '789':'07-finance/07-06-pl-model','367.08':'07-finance/07-06-pl-model',
      '156.52':'07-finance/07-06-pl-model','108.34':'06-service/06-04-retention',
      '34.65':'06-service/06-04-retention','1,908':'03-supply/03-04-order-calculator',
      '1,057.8':'03-supply/03-04-order-calculator','2,860':'05-sales/05-02-01-kickstarter-deposit',
      '71,500':'05-sales/05-02-01-kickstarter-deposit'}
docs={}
for p in pathlib.Path('docs').rglob('*.md'):
    s=str(p)
    if '/en/' in s or '/public/' in s or '.vitepress' in s: continue
    docs[s[5:-3]]=p.read_text(encoding='utf-8')
for val,src in CORE.items():
    hits=[r for r,t in docs.items() if val in t and r!=src]
    print(f'{val:9} owner={src.split("/")[-1]:34} others {len(hits)}: {hits[:4]}')
PY

What the human does: check meaning, not just that the digits match.

Why this cannot be fully automated: three approaches failed —
"Citation + any number in context" → 1002 hits, ~90% noise
"Citation + relation word + number" → 97 hits, still full of index tables
"Citation + numbers in that sentence + compare against target" → 163 hits, but one sentence often cites several articles, so attribution is unclear
The workable route was "pin core numbers → find citation points → verify basis by hand", narrowing to 34 checkable spots.

Layer 4: table internal consistency

What to check: within one table, do the row-by-row additions and subtractions equal the subtotal rows?

What the machine does: all of it, provided the formula relations are explicit — already encoded as 105 assertions.

bash

node scripts/check-numbers.mjs      # 105 assertions across 5 key tables

Tables covered:

Table	What is verified
07.06 P&L master table	5 chains × 3 scenarios = 15 items
03.05 PSI yield-loss chain	1,060 → 1,007 → 1,002 → 1,000
03.04 first-batch cash	three components plus NRE, including low/high bounds
06.03 RMA five accounts	share / MSRP% / amount totals plus each line
05.02.01 crowdfunding funnel	4 waves → backers / revenue / weighted ASP

Lesson: verifying a settlement table must be done row by row. Checking only the EBIT line once let two errors in the gross-margin line (wrong formula relation, negative signs added as positive) cancel each other out, so the final line "looked right" and the whole table was wrongly judged correct.

3. Handling gate failures

Message	Usual cause	Fix
`H1 contains ~ / _ separators`	Feishu export escaping not restored	Fix the local H1, then push to Feishu to sync the node title
`table separator column count mismatch`	Hand-written table missing a column	Add the missing pipe; columns must equal the header
`table split by a blockquote`	A `>` note inserted mid-table	Move the note below the table
`checklist item restates the heading`	Auto-generator output	Rewrite as an executable action
`ZH/EN structural mismatch`	Only one side edited	Fill in the other side, or confirm the difference is intentional
`metric threshold restated outside its owner`	Non-authoritative article wrote value + comparator	Change to "see 00.02 §5", or register in `EXEMPT_LINES`
`number recomputation failed`	Parameter changed but conclusion did not	Update the conclusion; if the parameter itself is wrong, fix the parameter
`identical body text`	Batch script wrote the wrong file	Restore from git history, see below

Three mistakes already made here

Trap 1: Feishu strips HTML comments

Never use  as an inline exemption. Feishu's markdown import drops HTML comments, so after one sync the marker is gone and CI fails unexpectedly. Register exemptions in the script's EXEMPT_LINES (version-controlled).

Trap 2: "the batch script finished" is not "it wrote the right thing"

An emoji-removal script once overwrote the entire body of 03.01 with the body of 01.07. The page existed, links resolved, the build passed — but the content was gone. The duplicate-body-fingerprint gate now catches this.

Batch scripts must verify "file → expected content" item by item.

Trap 3: push to Feishu first, then sync

Reverse the order and the stale Feishu copy overwrites local edits.

bash

# Correct order
python3 /tmp/push_doc.py docs/<article>.md <obj_token>   # 1. push to Feishu
node scripts/sync-feishu.mjs                             # 2. pull back and verify
git add -A && git commit && git push                     # 3. commit and deploy

4. When to stop

Once all four layers have run, only two kinds of problem remain, and neither is in scope for this playbook:

Kind	Example	Why the machine cannot do it
Business judgement	Should the gross-margin redline be 35% or 50%? Is an NPS of 50 a sensible bar?	No right answer, only trade-offs
External data truth	Indonesia's actual VAT rate, platform fee schedules	Needs local network access and official channels

The rule of thumb: if the question is "where did this number come from", it is external data; if it is "what should this number be", it is business judgement. Neither should be expected of an automated check.

5. Audit records

Round-by-round findings and fixes live in AUDIT-CHANGELOG at the repository root. Entries are numbered 3.1 to 3.31, each recording "what was found, how it was fixed, which gate was added".

Content Audit Playbook ​

1. What the five gates actually cover ​

2. Four audit layers and their commands ​

Layer 1: heading versus body ​

Layer 2: table versus conclusion ​

Layer 3: cross-article number citations ​

Layer 4: table internal consistency ​

3. Handling gate failures ​

Three mistakes already made here ​

4. When to stop ​

5. Audit records ​

Content Audit Playbook

1. What the five gates actually cover

2. Four audit layers and their commands

Layer 1: heading versus body

Layer 2: table versus conclusion

Layer 3: cross-article number citations

Layer 4: table internal consistency

3. Handling gate failures

Three mistakes already made here

4. When to stop

5. Audit records