@yadsh/dsh-answer-review-gate

Independent answer review gate for DeepSeek Harness agents: before a candidate final answer is allowed to complete its turn, an independent reviewer checks it and can send it back for correction. The review is a lifecycle requirement enforced at the turn boundary — not a line in the system prompt.

What it does

User → primary agent → research / tools / background subagents
     → candidate answer
     → answer-review-gate → reviewer
            ├─ PASS  → the turn may finish
            └─ REVISE → findings are steered back into the primary,
                        which produces a corrected candidate that is
                        reviewed again (bounded rounds)

• Interim orchestration turns are never reviewed: while the session's own background delegations (continuable subagents, one-shot background jobs) are still pending, a turn close is treated as a pause, not a final answer. The decision is pure runtime state — the plugin records delegated child ids from the structured delegation tool result and clears them when the runtime's subagent-settled notice arrives. No text heuristics ("waiting for…") participate in this decision.
• A review PASS applies to the exact candidate content (SHA-256 over the normalized text). Any later edit invalidates the PASS and the new candidate is reviewed again; the receipt follows the user request, so an unchanged candidate is never reviewed twice.
• Review rounds are bounded per user turn (maxReviewRounds, default 3). The budget belongs to the user request the candidate answers — the agent turns and revisions that request takes share one budget, and no agent turn hands out a fresh one.
• A revision steer admits exactly one visible artifact: the corrected answer. Findings are handed back as a delimited <review_notes> block, and the steer asks for no account of the review — a demand to "state the disproof" of a rejected objection was resolved literally on a live stand, and the user read the primary's argument with its reviewer. A draft that opens by disputing the review (Опровержение вывода ревизора: …) is that leak: in a turn that has already been reviewed the gate demands the answer's shape once, and that draft is not handed to a reviewer, so it cannot be certified as verified. The honest bounds: a committed message cannot be edited at this seam, so a second such draft goes through the ordinary review path, and a request the user waived is settled before the guard runs.
• The reviewer itself is exempt: it runs as a subagent child and subagents are never gated, so the reviewer cannot recursively review itself.

Explicit one-request waiver

When deployment policy permits it, a user can send one request without the automatic reviewer:

/no-review <request>

The command may carry the same image or file attachments as an ordinary request. It immediately shows a notice that the answer will not be independently verified, then submits <request> as the real user message. The waiver applies to that request only.

This is a structured command contract, not natural-language detection. The successful command/done.sourceEventSeq must point to the durable agent/inbox/spliced event that admitted the exact user-message id, and the same lifecycle must start with a human-issued command/run for no-review. Phrases such as “review is not needed”, including quoted or negated mentions, never bypass the gate. A failed command, unrelated message, or old command lifecycle does not bypass it either.

Reviewer backends

reviewer.backend: domain-expert (default)

Runs the reviewer domain configured in dsh-domain-experts by stable domain id (reviewer.domain, default answer-reviewer), so the reviewer persona, policy, tools and model are tuned in the domain-experts UI, not here. The verdict is derived from the expert's own structured protocol: findings and conflicts are the review objections; a run with no findings and no conflicts passes. Phase 1 bound: the domain-experts test entry owns its own lifecycle, so the reviewer run is not bound to the gate's abort signal (it is a bounded single run).

reviewer.backend: subagent

Starts a native reviewer child (reviewer.provider, default spawn) with a fresh context: it receives only the user request and the candidate answer, an optional persona (reviewer.persona), a read-only tool allow-list (reviewer.allowedTools; unset means the read-only set, an explicitly empty list means no tools) and an optional model route (reviewer.route / reviewer.model / reviewer.reasoningEffort). The verdict is requested as structured JSON; unparseable output is a reviewer failure, not a pass. For correlated-failure reduction, route the reviewer to a different provider or model than the primary agent.

Failure policy (failMode)

What happens when the review could not complete — reviewer failure, malformed verdict, or rounds exhausted:

• open — allow the answer; audit the failure.
• warn (default) — allow the answer, but steer an instruction requiring the primary to explicitly qualify that independent verification did not complete.
• closed — steer a demand to revise the answer or end with an explicit unverified-answer notice; the answer is never presented as verified.

A reviewer failure is never converted into a PASS. At most one failure steer is issued per user turn, so the loop cannot livelock. Note the honest bound: a plugin cannot hard-block a turn at this seam — closed mode forces a revision-or-disclaim step rather than silently dropping the answer.

Configuration

enabled: true
reviewer:
  backend: domain-expert
  domain: answer-reviewer
  # backend: subagent
  # provider: spawn
  # route: ""
  # model: ""
  # reasoningEffort: ""
  # persona: ""
  # allowedTools: []          # unset means the read-only set; [] means no tools
maxReviewRounds: 3
failMode: warn
trackBackgroundDelegations: true
minCandidateChars: 80
excludedAgents: []
waiver:
  enabled: true
  allowedInClosedMode: false
audit:
  enabled: true
  maxEntries: 500

Field	Meaning
enabled	Master switch; false registers no listeners at all.
reviewer.backend	domain-expert or subagent.
reviewer.domain	Reviewer domain id in dsh-domain-experts (stable id, not a display name).
reviewer.provider	Subagent provider name for the subagent backend.
reviewer.route / reviewer.model / reviewer.reasoningEffort	Reviewer route overrides for the subagent backend.
reviewer.persona	Persona instruction for the subagent reviewer.
reviewer.allowedTools	Read-only tool allow-list. Unset means the read-only set (read, read_image, glob, grep, docs_read, docs_search); an explicitly empty list means the reviewer works without tools. Destructive and writing names are removed whatever the list says.
maxReviewRounds	Review/revision rounds per user turn before the failure policy applies (1–10).
failMode	open / warn / closed, see above.
trackBackgroundDelegations	Suppress review while the session's background work is pending.
minCandidateChars	Candidates shorter than this skip review.
excludedAgents	Session-id substrings that are never reviewed (reviewer children are exempt structurally).
waiver.enabled	Register and honor /no-review <request> for exactly one request.
waiver.allowedInClosedMode	Permit the command while failMode: closed; default false.
audit.enabled / audit.maxEntries	In-memory audit ring of decisions (metadata only — never prompt or response text).

Requirements

• DeepSeek Harness >=0.1.7-rc.2 <0.2.0
• Node.js ^22.19.0 or >=24.0.0

Installation

dsh plugin --profile <profile> add @yadsh/dsh-answer-review-gate

The --profile flag is required.

Compatibility

• DeepSeek Harness >=0.1.7-rc.2 <0.2.0 (see compatibility.json).
• Uses the commands host service plus the agent/turn-stopping, tools/result and agent/inbox/inserted lifecycle seams — no DSH core changes.

Security model

• The reviewer runs in a fresh context and receives only the review material (user request, candidate answer). It does not inherit the primary agent's hidden reasoning.
• The domain-expert backend's tool access is whatever the configured domain grants; configure the reviewer domain read-only. The subagent backend's tools are reviewer.allowedTools minus the names src/reviewer-tools.ts excludes: an unset list gives the reviewer the read-only set (reads and searches), and a listed file_delete, write, edit, apply_patch, str_replace_editor, bash, shell, run_code, lsp, dsh_lightrag_delete or a terminal_/job_ tool is dropped before the child is started. A reviewer that reaches for a deletion stops the turn it belongs to: on a QA stand the call parked above the parent's composer for an approval a delegated child can never receive, so the boundary is held both by the allow-list and by the surface, which refuses a delegated call outright.
• Reviewer output is untrusted model output: it is parsed and validated, a malformed verdict is a reviewer failure handled by the failure policy, and the primary remains responsible for checking reviewer evidence rather than blindly obeying it.
• Audit records contain decision metadata only (session, turn, candidate hash, rounds, durations, verdict or structured waiver reason) — no prompt or response text. /no-review also sets recordInput: false, so its lifecycle record does not duplicate the request text; the normal user message remains the sole authoritative copy.

Scope of Phase 1

Shipped: turn-stopping gate, both reviewer backends, candidate hashing, REVISE-via-steer, round limit, recursion protection, background-delegation suppression, structured one-request waiver, in-memory audit, tests. Deferred to later phases: durable /replayable review state, evidence bundle, full metrics export, and a UI badge.

See SPEC.md for the product contract.

Development

pnpm build
pnpm lint
pnpm typecheck
pnpm test

License

MIT