Agentic Knowledge Reports

​

Motivation / problem

1. Governance questions that the graph already knows the answer to. “Is this doc canonical? Orphaned in the graph? Superseded? What’s its evidence tier?” are deterministic facts in the canonical layer, not LLM judgments. Asking an LLM to guess them is slower, costs money, and can hallucinate.
2. Anti-hallucination on the extraction itself. A green flag means the model was confident — not that the value is actually supported by the cited evidence.

​

Theory & background

• extract — today’s behaviour, unchanged and the default. One RAG single-shot per (doc, column): retrieve, ask the LLM, flag + cite. Every pre-v8.19 review is byte-identical (the field is absent → extract).
• graph — a deterministic, LLM-free governance metric resolved from the canonical graph + document columns. No model call, no cost, no hallucination — the value is a fact (is_canonical = true, incoming_edges = 4, …). The graph agent wins over the json_path shortcut when both are set.
• verify — a bounded anti-hallucination second pass. After an extract produces a value, a verify call re-checks it against the document’s cited evidence and can only downgrade the flag (green → yellow / else → red) when the value isn’t supported. It is never worse than extract: a verify-call failure keeps the original cell (R14).

​

Design

​

Data model / contract

• PHP — TabularReviewExtractor + GovernanceColumnResolver + the BuiltInWorkflowSeeder library.
• HTTP — the existing api/admin/tabular-reviews/* (list / show / create / generate), now accepting agentic columns.
• MCP — App\Mcp\Tools\KbRunReportTool on the enterprise-kb server (read a saved report’s matrix, tenant-scoped R30, OFF-path safe R43), bounded by COUNT(DISTINCT) + LIMIT so an agent can’t pull an unbounded matrix.

​

Security & flags (R32 / R30 / R43)

• RBAC. Every tabular-review route is behind can:viewTabularReviews / can:manageTabularReviews, R32-matrix-locked. The agentic upgrade adds no new route group — it extends the existing controllers’ request contracts.
• Tenant isolation (R30). GovernanceColumnResolver scopes every kb_edges / document query to the active tenant; KbRunReportTool filters the report + cells by tenant. A client cannot read another tenant’s report or widen scope through a filter.
• OFF-path safe (R43). KbRunReportTool returns a well-formed empty payload when the report/cells are absent rather than throwing; verify failures and graph unknowns degrade to a grey/red cell (R14) instead of erroring the whole generate.

​

Decision rationale (ADR-style)

• A per-column agent dimension, not a report-level mode. Governance audits mix deterministic facts and grounded judgments in the same grid; a whole-report mode would force a false choice. Making agent a column property lets one report carry graph facts next to extract/verify answers. See architecture decisions.
• Deterministic graph metrics over LLM guesses. Canonical status, edge degree, orphan-ness and supersession are facts in the graph. Resolving them with GovernanceColumnResolver (no model call) is faster, free, and cannot hallucinate — and it reuses the real EvidenceTier/CanonicalStatus enums so the taxonomy never drifts (R9).
• verify can only downgrade. An anti-hallucination pass that could raise a flag would itself be a hallucination risk. Constraining it to downgrade-only (and keeping the original cell on failure) makes the verified run monotonic and never worse than the extraction (R14).
• Reuse workflows for the library. The ready-made “precotte” templates are just system-owned tabular workflows — no new model, no new admin surface, and they show up in the existing workflow gallery for free.
• Glide canvas grid deferred. The accessible DOM matrix (per-cell testids + ARIA, R11/R15) ships now; the canvas grid + SSE progressive-paint are documented v8.19.x follow-ups because canvas cells can’t carry per-cell testids/ARIA — testability and a11y come first.

​

Worked example

-- a graph governance cell — a fact, not a guess
SELECT column_index, content->>'summary' AS value, content->>'flag' AS flag
FROM tabular_cells
WHERE review_id = 7 AND column_index = 3   -- the "is_orphan" column
ORDER BY document_id LIMIT 1;
-- 3 | "orphan" | "red"   (canonical doc with zero edges)

extract → "Yes, the policy mandates 30-day retention" (green)
verify  → the cited chunk says "retention is configurable", not "30-day"
result  → flag downgraded to yellow; original value kept (R14)

{ "tool": "run-report", "input": { "review_id": 7, "max_rows": 50 } }
// → { "summary": { "documents": 42, "total_documents": 213 }, "columns": [ { "metric": "is_orphan", ... } ], ... }

​

Gotchas & operations

• graph needs a metric. A graph column without a governance metric is a misconfiguration — the FE gates submit on it and the BE validator rejects it (required_if), so a bad column can’t reach generation and 422.
• graph wins over json_path. If a column sets both, the deterministic graph resolver runs; json_path is the LLM-free shortcut for extract-family columns.
• verify adds a bounded second call. It is the one agent kind that costs extra per cell (one verify call after the extract). Use it for the columns where anti-hallucination matters, not blanket-on every column.
• Existing reviews are unchanged. A pre-v8.19 review has no agent on its columns → every column is extract → byte-identical output. The agentic upgrade is purely additive (R27).
• Re-run after canonical changes. graph cells reflect the canonical graph at generate time; promote/supersede/delete a doc and re-generate to refresh the governance matrix.